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NAS 
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National  Technical  Information  Ser¬ 
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“P-value.”  A  statistical  symbol  indi¬ 
cating  the  probability  of  obtaining  a 
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in  100  (/>  <  .05),  1  time  or  less  in  100 
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<  .001)  independent  samples 

Professional  Air  Traffic  Controller 
Organization 

Statistical  symbol  for  the  multiple 
correlation  between  a  single  crite¬ 
rion  (Y)  and  multiple  predictors  (X^, 
X2,  ...  X).  R  may  range  between  0 
and  1,  with  1  indicating  that  Y  is 
perfectly  predicted  by  some  weighted 
combination  of  Xs.  In  multiple  re¬ 
gression  analysis,  the  values  on  one 
or  more  predictors  (X^,  X^, ...  X)  are 
used  to  predict  or  estimate  values  on 
the  criterion  (Y).  Each  predictor  is 
statistically  weighted  by  the  stan¬ 
dardized  regression  coefficient  (b) 
such  that  overall  errors  in  the  predic¬ 
tion  of  Y  values  are  minimized.  The 
degree  of  overall  linear  relationship 
between  Y  and  the  predictors  X^,  X^, 
...  X.  is  indexed  by  the  multiple  cor¬ 
relation  {R).  An  R  of  1.0  indicates 
that  Y  is  perfectly  predicted  from  the 
combination  of  predictors,  and  an  R 
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dicted  from  any  of  the  predictors. 
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INTRODUCTION 


Dana  Broach 

FAA  Civil  Aeromedical  Institute,  Oklahoma  City,  Oklahoma 


On  August  3, 1 98 1 ,  the  majority  of  the  nation’s  air 
traffic  controllers  went  on  strike.  On  August  5, 1981, 
President  Reagan  summarily  fired  the  controllers 
who  had  not  returned  to  work.  The  controller  strike 
left  4,669  operational  controllers  available  in  August 
1 98 1  to  staff  the  nation’s  Air  Route  Control  Centers, 
terminal  radar  approach  control,  and  airport  control 
towers  (National  Transportation  Safety  Board,  1981). 
The  Federal  Aviation  Administration  (FAA)  con¬ 
fronted  a  challenge  unique  in  the  annals  of  American 
government:  that  of  rebuilding  a  core,  technical, 
highly  trained  workforce  in  an  occupation  directly 
responsible  for  the  safety  of  millions  of  lives  on  a  daily 
basis.  From  late  calendar  year  1981  to  mid-year 
1995,  the  FAA  rebuilt  this  critical  workforce  through 
a  large-scale  testing,  screening  and  training  program. 

The  testing  and  screening  programs  operated  from 
1981  through  March,  1992.  Field  training  data  col¬ 
lection  ended  in  the  summer  of  1995.  By  then,  the 
controller  workforce  was  near  its  pre-strike  level, 
with  about  14,400  controllers  working  in  the  nation’s 
en  route  centers,  terminal  radar  facilities,  and  airport 
towers.  Throughout  this  period,  the  Civil  Aeromedi¬ 
cal  Institute  conducted  a  longitudinal  research  pro¬ 
gram  focused  on  the  validation  of  the  air  traffic 
control  specialist  (ATCS)  selection  program. 

The  following  papers  represent  the  first  major 
retrospective  analysis  of  the  complete  data  set 
describing  the  recovery  of  the  FAA’s  en  route  and 
terminal  ATCS  workforce  following  the  1981  con¬ 
troller  strike. 


In  the  first  paper.  Jay  Aul  describes  the  personnel 
processes  supporting  the  recovery  of  the  controller 
workforce,  focusing  on  recruitment  and  hiring  pro¬ 
grams  for  the  center  and  terminal  options. 

Second,  a  detailed  description  of  the  numbers  of 
persons  tested  and  the  instrument  used  to  test  for 
aptitude  are  presented  by  Dana  Broach.  The  second 
paper  also  evaluates  the  validity  of  the  test  battery  as 
a  predictor  of  performance  at  the  FAA  Academy  and 
in  the  field. 

The  third  paper,  by  Pam  Della  Rocco,  offers  a 
definitive  statistical  portrait  of  the  1981  to  1992  FAA 
Academy  screening  programs  for  the  en  route  and 
terminal  options.  Della  Rocco  also  evaluates  the 
validity  of  the  FAA  Academy  screening  programs  as 
predictors  of  field  training  outcomes. 

Fourth,  the  on-the-job  training  (OJT)  program  in 
en  route  and  terminal  field  facilities  through  June, 
1995,  is  described  by  Carol  Manning.  These  papers, 
taken  together,  provide  a  definitive  description  of  the 
processes  used  to  test,  screen,  and  train  persons  for 
the  ATCS  occupation  between  1981  and  1992. 

The  fifth  paper,  by  Richard  Thompson,  chronicles 
air  traffic  controller  perceptions  of  organizational 
climate  in  which  this  recovery  occurred,  drawing  on 
FAA  survey  data  and  documented  organizational 
initiatives. 

Finally,  the  demographics  of  the  current  ATCS 
workforce,  projections  about  future  trends  and  pos¬ 
sible  requirements,  and  potential  areas  of  research  in 
relation  to  both  demographic  and  technological 
trends,  are  described  by  David  Schroeder,  Dana 
Broach,  and  Bill  Farmer. 
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CHAPTER  1: 


EMPLOYING  AIR  TRAFFIC  CONTROLLERS,  1981-1992 

Jay  C.  Aul 

FAA  Office  of  Personnel,  Washington,  District  of  Columbia 


The  civil  service  procedures  and  processes  used  to  recruit  and  select  air  trajftc  control  specialists  (ATCS)  following 
the  1981  controller  strike  are  described.  Recruitment  approaches  and  strategiesy  basic  occupational  qualifications 
requirements  for  applicants  applying  under  competitive  rules,  and  additional  selection  criteria  such  as  an  interview, 
medical  examination,  and  security  and  suitability  investigation  are  presented.  Dijferent  noncompetitive  employment 
processes,  such  as  cooperative  education,  are  briefly  surveyed,  concluding  with  current  trends  in  controller  selection. 


On  August  3,  1981,  10,438  members  of  the  Pro¬ 
fessional  Air  Traffic  Control  Organization  (PATCO) 
staged  a  job  action,  or  strike,  against  the  Federal 
government.  On  August  5,  1981,  President  Reagan 
summarily  fired  the  controllers  who  had  not  returned 
to  work.  The  PATCO  strike  left  4,669  operational 
controllers  available  in  August  1981  to  staff  the 
nation’s  airport  control  towers,  terminal  radar  ap¬ 
proach  control  facilities,  and  Air  Route  Traffic  Con¬ 
trol  Centers  (National  Transportation  Safety  Board, 
1981).  The  Federal  Aviation  Administration  (FAA) 
needed  to  rebuild  as  quickly  and  prudently  as  pos¬ 
sible  after  the  loss  of  more  than  half  of  its  controller 
workforce.  As  the  FAA’s  air  traffic  controllers,  more 
properly  called  air  traffic  control  specialists  (ATCSs), 
are  civil  servants,  the  employment  process  was  gov¬ 
erned  by  United  States  federal  civil  service  laws  and 
regulations.  The  regulatory  agency  overseeing  these 
laws  and  regulations  was  the  U.S.  Office  of  Personnel 
Management  (OPM).  The  rules  and  regulations  re¬ 
sulted  in  well-defined  and  rather  strict  employment 
procedures  to  ensure  that  there  was  open  and  fair 
competition  for  controller  positions  in  the  FAA.  The 
purpose  of  this  paper  is  to  provide  an  overview  of  the 
employment  procedures  used  by  the  FAA  to  recruit  and 
select  ATCSs  during  the  recovery  from  the  1981  strike. 

Recruitment 

The  process  logically  began  with  finding  and  at¬ 
tracting  controller  candidates.  Under  civil  service 
rules,  job  openings  must  be  advertised  in  a  way  that 
ensures  open  competition.  The  government  calls 
these  advertisements  “vacancy  announcements.”  In 


concrete  form,  they  are  printed  documents  that  con¬ 
tain  certain  detailed  information,  disclaimers,  and 
statements.  The  announcements  must  also  be  distrib¬ 
uted  to  organizations,  such  as  state  employment 
services.  Advertising  was  not  only  a  legal  require¬ 
ment,  it  was  a  practical  necessity  during  the  recovery 
period.  The  traditional  sources  of  candidates  were 
former  military  controllers  (Collins,  Manning,  & 
Taylor,  1984),  and  people  who  knew  about  the  job 
from  some  kind  of  personal  contact.  That  contact 
often  turned  out  to  be  relatives  or  neighbors  who 
were  air  traffic  controllers.  Those  sources  were  insuf¬ 
ficient  to  rebuild  the  work  force  after  the  1981  strike. 
To  hire  enough  people,  it  was  necessary  for  the  FAA 
to  recruit  from  the  general  population.  Advertising 
also  had  to  overcome  the  perception  that  the  strike 
circumstances  did  not  make  the  job  the  of  an  ATCS 
sound  appealing.  The  FAA  and  controller  jobs  re¬ 
ceived  much  negative  publicity  in  the  years  following 
the  strike.  Moreover,  the  FAA  was  competing  for  a 
scarce  resource  consisting  of  people  under  age  31  who 
could  score  high  on  the  civil  service  examination  for  air 
traffic  control  positions.  The  military  and  private  sec¬ 
tors  also  valued  and  tried  to  attract  bright  young  people. 

The  first  post  strike  vacancy  announcements  were 
little  different  from  those  issued  by  other  government 
agencies.  They  were  several  pages  printed  in  black  ink 
on  white  paper,  and  the  only  graphic  was  the  agency 
logo.  There  was  an  attempt  shortly  after  the  strike  to 
create  a  more  attractive,  exciting  announcement. 
That  turned  out  to  be  a  picture  of  an  airliner  taking 
off,  with  a  banner  above  it  saying  something  like, 
“Control  the  Skies.”  Unfortunately,  the  picture  was 

Preceding  Page  Blank 
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printed  with  red  ink.  With  the  smoke  or  dust  trailing 
the  aircraft,  it  looked  to  some  people  as  if  the  airliner 
was  in  trouble.  That  particular  format  for  the  an¬ 
nouncement  was  dropped.  At  the  same  time,  for 
budget  reasons,  there  were  government-wide  restric¬ 
tions  on  printing  in  multiple  colors.  Eventually, 
however,  the  FAA  received  permission  to  print  its 
announcement  as  a  four-color  glossy  brochure.  The 
brochure  was  quite  different  from  the  standard  gov¬ 
ernment  vacancy  announcement.  In  addition  to  the 
required  information,  it  contained  a  brief  story  out¬ 
lining  how  controllers  handle  a  single  airline  flight. 
There  were  sample  items  from  the  civil  service  exami¬ 
nation  for  air  traffic  controllers  and  color  pictures  of 
controllers  at  work  in  various  settings.  The  FAA  built 
a  staff  of  full  time  recruiters  who  sought  out  sources 
of  applicants.  The  FAA  learned  to  use  other  recruit¬ 
ment  methods  during  this  period  as  well.  FAA  field 
personnel  offices  frequently  used  newspaper  adver¬ 
tisements  to  recruit.  Magazine,  television,  and  radio 
advertisements  were  also  used  in*  some  markets 
(Schultz  &  Marshall-Mies,  1988). 

Selection 

Competitive  processes 

Basic  qualifications  and  pre-hire  testing.  One 
reason  that  extensive  recruitment  was  necessary  was 
that  potential  controllers  had  to  meet  a  number  of 
qualifications  requirements.  First,  they  had  to  be 
United  States  citizens.  Second,  in  most  cases,  they 
could  be  no  younger  than  1 8  years  of  age.  They  could 
not  have  reached  their  31st  birthday  prior  to  initial 
appointment  into  the  terminal  or  en  route  options. 
To  qualify  for  consideration  at  the  entry  level  civil 
service  grade  5  (“GS-5”),  an  applicant  needed  some 
combination  of  undergraduate  or  graduate  level  edu¬ 
cation,  progressively  responsible  work  experience, 
and/or  specialized  aviation  experience.  Candidates 
were  also  required  to  take  (and  pass)  the  OPM  civil 
service  examination  for  air  traffic  control  specialists. 
This  written  test  was  designed  to  identify  people  with 
the  aptitude  to  be  air  traffic  controllers  (Broach,  this 
volume).  Under  Title  5  of  the  U.S.  Code  of  Federal 
Regulations,  all  such  civil  service  entrance  examina¬ 
tions  must  be  scored  on  a  scale  of  100.  The  bottom 
point  is  not  designated  in  the  regulations.  The  passing 
score  for  a  civil  service  test  must  be  set  at  70.  To  avoid 
adverse  impact  against  minority  groups  and  women, 


OPM  required  that  the  score  of  70  on  this  test  be 
calibrated  to  the  50th  percentile  of  the  population 
applying  for  these  jobs  at  the  GS-5  grade  level. 

However,  most  recruitment  was  for  grade  7  (“GS- 
7”),  which  was  the  more  common  entry  grade  for  the 
controller  occupation.  The  qualifying  score  on  the 
written  ATCS  aptitude  test  for  the  GS-7  grade  level  was 
75. 1  for  applicants  without  prior  specialized  experience 
or  education.  To  qualify  for  GS-7  with  a  score  of  70,  an 
individual  had  to  have  additional  experience  or  educa¬ 
tion.  Typically,  this  was  certain  kinds  of  aviation  expe¬ 
rience,  one  year  of  graduate  school,  or  superior  grades  in 
college.  Federal  law  also  required  that  extra  points  be 
given  to  certain  military  veterans.  The  matter  of  who 
received  these  extra  points  is  too  complex  to  fully 
explain  here.  For  simplicity,  it  can  be  explained  as 
including  veterans  of  the  U.S.  military,  who  served  in 
declared  wars,  or  in  military  actions  for  which  campaign 
badges  or  expeditionary  medals  were  awarded.  Nor¬ 
mally,  five  extra  points  were  added  onto  the  examina¬ 
tion  score  for  veteran’s  meeting  these  criteria.  If  the 
veteran  had  a  military  service-related  disability,  he  or  she 
received  ten  extra  points,  instead  of  five.  Veterans  also 
had  priority  in  hiring  over  non-veterans.  For  example,  a 
candidate  with  veteran’s  preference  had  priority  in 
hiring  over  a  candidate  with  the  same  score,  but  without 
the  veteran’s  preference. 

Interview,  medical,  and  security  criteria. 

Candidates  were  also  interviewed.  The  interview  had 
several  purposes.  One  key  purpose  was  to  provide  infor¬ 
mation  to  the  candidates  about  the  ATCS  occupation. 
Most  candidates  did  not  have  a  clear  understanding  of 
the  work  of  air  traffic  controllers.  It  was  important  to 
explain  the  concept  of  rotating  shift  work,  because  many 
did  not  understand  it.  It  was  also  a  chance  for  the 
interviewer  to  evaluate  the  candidate  for  such  factors  as 
ability  to  speak  clearly  and  to  speak  in  English.  Inter¬ 
viewers  were  asked  to  make  a  hiring  recommendation 
for  each  candidate,  although  the  interview  was  rarely 
used  to  screen  out  candidates. 

Candidates  who  progressed  through  these  hurdles 
then  had  to  meet  the  medical  requirements  for  these 
positions.  The  medical  standard  for  entry  into  these 
positions  was  (and  is)  quite  strict.  It  included  vision  and 
hearing  standards,  as  well  as  a  form  of  mental  health 
screen.  Personality  assessment  has  been  used  formally  by 
the  FAA  in  conjunction  with  the  ATCS  programs  since 
1965  (Convey,  1984).  From  the  outset,  the  38-item 


4 


Chapter  7;  Employing  Air  Traffic  Controllers^,  1981-1992 


subset  from  the  two  187-item  parallel  forms  of  the 
Sixteen  Personality  Factor  Questionnaire  (1 6PF;  Cattell, 
Eber,  &  Tatsuoka,  1970)  was  intended  to  operate  as  an 
indicator  of  emotional  instability.  Candidates  flagged 
by  the  38'item  scale  were  referred  for  psychiatric  and 
psychological  evaluation. 

Finally,  the  candidates  had  to  pass  a  background 
investigation.  This  investigation  was  used  for  two  pur¬ 
poses.  One  was  to  determine  if  the  person  would  receive 
a  security  clearance.  The  other  was  to  determine  the 
person’s  general  suitability  for  federal  employment. 
Individuals  who  had  been  found  guilty  of  crimes,  who 
had  been  discharged  from  the  military  under  less  than 
honorable  conditions,  who  had  been  fired  with  cause 
from  previous  employers,  or  who  had  been  convicted  of 
any  alcohol  or  drug-related  offense  had  great  difficulty 
being  cleared  under  this  process.  The  FAA  also  looked 
with  caution  on  people  who  had  repeated  minor  viola¬ 
tions,  such  as  multiple  speeding  tickets. 

Large  numbers  of  people  were  processed  under 
these  procedures.  At  peak,  nearly  50,000  people  were 
tested  in  a  year.  During  the  first  years  after  the  strike, 
the  FAA’s  personnel  offices  were  required  to  recruit 
and  hire  specific  numbers  of  people  for  each  new  class 
date  at  the  FAA  Academy.  These  numbers  were  set 
ambitiously  high;  initially,  therefore,  a  few  people 
with  test  scores  as  low  as  70  were  sent  to  the  FAA 
Academy.  Over  time,  the  FAA  was  able  to  recruit  and 
hire  people  with  increasingly  higher  test  scores.  Dur¬ 
ing  the  late  1980s  until  1992,  the  minimum  hiring 
scores  varied  from  about  85  to  about  95. 

Post-hire  screening.  Once  someone  met  the  basic 
eligibility  criteria,  passed  the  OPM  test,  was  medi¬ 
cally  cleared,  passed  a  background  investigation,  and 
was  interviewed,  the  person  was  usually  hired.  How¬ 
ever,  the  selection  process  did  not  stop  there.  Select¬ 
ees  reported  to  the  FAA  Academy  screening  program 
in  Oklahoma  City,  Oklahoma  (Della  Rocco,  this 
volume).  The  individuals  were  officially  appointed 
into  the  civil  service  at  the  FAA  Academy,  becoming 
paid  employees  of  the  FAA.  They  spent  the  first 
several  weeks  of  their  employment  in  the  FAA  Acad¬ 
emy  screening  program.  The  FAA  Academy  program 
was  considered  to  be  initial  training,  but  was  con¬ 
ducted  with  an  emphasis  on  screening  out  those 
unlikely  to  be  successful  in  further  FAA  controller 
training  (Boone,  1984).  The  last  version  of  the  screen¬ 
ing  program  was  approximately  nine  weeks  long. 
Those  who  failed  the  FAA  Academy  program  usually 
had  their  employment  terminated.  The  pass  rate 


varied,  but  was  about  60  percent  for  most  of  the 
period  it  operated.  On  average,  about  30  percent 
failed  outright,  and  another  10  percent  withdrew 
from  the  program.  This  meant  that  nine  weeks  after 
starting  with  the  FAA,  approximately  40  percent  of 
the  new  hires  might  no  longer  be  employed  with  the 
agency. 

Re-engineering  the  selection  process.  The  process 
described  up  to  this  point  was  quite  lengthy.  In  1 988,  it 
was  estimated  to  take  on  average  18  months  to  hire  an 
entry  level  controller  to  send  to  the  FAA  Academy.  In 
particular,  it  might  take  several  months  to  obtain  a 
background  check  on  someone  or  to  medically  clear 
someone  who  required  a  special  review  of  his  or  her 
medical  situation.  As  a  result,  in  the  late  1980s  the  FAA 
embarked  on  what  today  would  be  called  a  “reengineering 
project.”  It  successfully  reengineered  the  hiring  process 
for  entry  level  controllers  and  effectively  reduced  the 
time  to  hire  from  18  months  to  about  six  weeks.  This 
was,  however,  late  in  the  post-strike  recovery  period  and 
many  people  still  experienced  delays  due  to  existing 
backlogs  of  work.  There  was  another  complicating 
factor  in  the  hiring  process.  A  number  of  companies 
discovered  a  market  for  teaching  people  to  take  the 
OPM  test.  These  companies  aggressively  marketed  their 
services  and,  as  a  result,  may  have  brought  many  more 
applicants  to  the  FAA.  Unfortunately,  we  also  know 
that  the  more  times  a  person  takes  the  test,  the  lower  the 
relationship  between  test  score  and  success  in  the  FAA 
Academy  screening  program  (Broach,  Young,  &  Farmer, 
1994;  Collins,  Nye,  and  Manning,  1990).  From  evi¬ 
dence  like  this,  we  concluded  that  the  test  was  subject  to 
practice  effects.  The  graduates  of  these  training  pro¬ 
grams  may  have  had  inflated  scores  on  the  test,  and  this 
could  have  contributed  to  some  increases  that  were  seen 
in  the  failure  rates  at  the  FAA  Academy  in  the  late  1 980s. 

Noncompetitive  processes 

So  far,  only  the  conventional,  competitive  hiring 
process  has  been  described.  Other  legitimate,  but  less 
well-  known  processes  were  also  used.  One  such  method 
was  cooperative  education.  The  FAA  formally  recruited 
students  at  colleges  in  certain  majors,  such  as  aviation, 
mathematics,  geography,  and  engineering.  As  employ¬ 
ees  in  the  FAA’s  cooperative  education  program,  the 
students  alternated  periods  of  training  to  be  controllers 
at  the  FAA  with  periods  of  study  in  college.  College 
graduates  who  met  all  the  requirements  described  pre¬ 
viously,  and  completed  the  cooperative  education  train¬ 
ing  program,  could  be  assigned  to  positions  as  entry 
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level  controllers.  They  then  attended  the  FAA  Academy 
screening  program.  In  addition,  the  FAA  moved  people 
internally  from  other  occupations  into  controller  jobs. 
Programs  were  established  that  allowed  lower-graded 
employees  to  take  the  OPM  written  ATCS  aptitude  test 
battery,  and  bid  for  entry-level  positions  in  air  traffic 
control.  Thus,  for  example,  a  secretary  or  clerical  worker 
could  compete  for  an  entry  level  training  position  as  a 
controller.  The  people  hired  under  this  program  usually 
started  as  controllers  below  the  normal  GS-7  entry 
grade.  They  received  additional  training  before  they 
reached  GS-7.  The  minimum  passing  scores  on  the 
OPM  test  still  applied  to  them,  and  they  were  required 
to  meet  the  same  medical  and  background  investigation 
requirements  as  other  applicants.  Of  course,  they  also 
had  to  pass  the  FAA  Academy  screening  program  upon 
reaching  the  GS-7  grade. 

Other  alternative  hiring  methods  centered  around 
former  military  controllers.  The  FAA  hired  many  former 
military  controllers  through  the  basic  civil  service  hiring 
process,  described  at  length  in  the  first  part  of  this  paper. 
It  was  also  not  uncommon  for  the  FAA  to  hire  military 
controllers  through  alternative  hiring  procedures.  For 
example,  Congress  established  special,  expedited  hiring 
mechanisms  for  Vietnam-era  veterans,  and  for  people 
who  had  recently  left  military  service.  The  FAA  still 
required  the  test,  medical,  and  background  investiga¬ 
tions  when  using  this  approach,  but  it  was,  nevertheless, 
somewhat  easier  to  hire  people  by  this  method.  Not 
uncommonly,  former  military  controllers  might  be 
hired  at  one  step  above  the  normal  entry  grade,  based  on 
their  military  controller  experience.  These  individuals 
could  bypass  the  FAA  Academy  screening  program  and 
go  directly  into  training  at  an  FAA  air  traffic  control 
facility.  In  1 988,  the  military  services  reduced  the  size  of 
their  forces.  The  FAA  took  advantage  of  the  opportu¬ 
nity  to  hire  military  controllers  (Manning  &  Aul,  1 992) . 
This  program  was  conducted  under  very  different  pro¬ 
cedures  as  authorized  by  OPM.  The  controllers  were 
hired  at  the  GS-9  grade,  one  grade  above  the  normal 
entry  grade.  Above  the  GS-7  grade,  the  OPM  test  was 
optional.  In  this  case,  the  agency  chose  not  to  use  it. 
Instead,  subject  matter  experts  (personnel  experts  and 
controllers)  rated  the  candidates’  experience,  education, 
and  training.  This  was  a  daunting  task,  as  there  were 
over  1,400  eligible  applicants.  A  standard  rating  guide, 
containing  six  factors,  was  used.  All  selectees  had  to 
meet  the  medical  requirements  and  be  cleared  through 
a  background  investigation.  In  addition,  FAA  air  traffic 
control  staff  contacted  the  selectees’  former  military 


facilities  to  verify  their  work  experience  and  obtain 
references  or  recommendations.  About  600  military 
controllers  were  hired  in  this  way.  They  were  placed 
directly  into  facility  training,  bypassing  the  FAA  Acad¬ 
emy  screening  program  (Manning  &  Aul). 

Current  Trends  in  Controller  Selection 

On  February  21,1 992,  the  FAA  closed  the  process  of 
hiring  from  the  general  population.  No  new  applica¬ 
tions  to  take  the  OPM  test  were  processed,  except  for 
certain  veterans  who  had  a  statutory  right  to  take  the 
examination.  On  August  12,  1993,  President  Clinton 
again  permitted  the  former  controllers  who  had  left  the 
agency  in  1981  to  be  considered  for  employment  with 
the  FAA.  The  FAA  accepted  applications  from  the 
former  controllers  from  September  1,  1993  through 
October  15, 1993.  Over  5,000  applications  were  received 
in  that  short  period.  From  1992  until  now,  it  has  not  been 
necessary  to  hire  significant  numbers  of  people.  Between 
1992  and  1 997,  the  FAA  hired  approximately  1 00  control¬ 
lers  peryear.  This  has  been  due  to  stable  workforce  with  very 
low  attrition  and  retirement  rates  (General  Accounting 
Office,  1997),  use  of  contractors  to  operate  less  busy  non¬ 
radar  towers,  and  a  slower  than  anticipated  growth  in 
aviation  in  the  early  part  of  the  decade. 

However,  the  FAA  may  increase  its  hiring  in  the  near 
future  but  under  very  different  conditions.  The  reason 
for  the  hiring  is  primarily  that  attrition  is  anticipated  to 
increase  in  the  next  five  years,  as  the  post-strike  genera¬ 
tion  of  controllers  ages.  The  FAA  is  now  starting  to  hire, 
to  ensure  that  replacement  staff  are  trained  and  ready 
when  needed.  The  FAA  estimates  that  about  500  new 
controllers  per  year  will  be  needed  over  the  next  several 
years.  The  General  Accounting  Office  (GAO)  has  re¬ 
cently  produced  slightly  lower  estimates  (GAO,  1997); 
but  both  organizations  point  to  the  need  for  future 
hiring  (Schroeder,  Broach,  &  Farmer,  this  volume). 
Approximately  50  cooperative  education  students  also 
began  work  study  programs  with  the  agency  in  1997. 
These  individuals  will  come  to  a  different  FAA.  In  April 
of  1 996,  the  FAA  was  legally  exempted  from  most  of  the 
civil  service  laws.  The  agency  now  has  the  authority  to 
develop  new  and  innovative  personnel  systems  and,  for 
many  months,  has  been  incorporating  new  procedures 
based  on  studies  of  the  best  practices  in  other  organiza¬ 
tions.  Based  on  those  best  practices  and  lessons  learned 
in  the  recovery  of  the  controller  workforce  between 
1981  and  1 992,  the  FAA  is  developing  and  implement¬ 
ing  innovative  processes  to  select  and  train  the  next 
generation  of  air  traffic  controllers. 
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Between  1981  and  1992,  entry  into  the  demanding  air  traffic  control  specialist  (ATCS)  occupation  in  the  Federal 
Aviation  Administration  (FAA)  was  determined  by  applicant  performance  on  a  written  aptitude  test  battery.  That 
A  TCS  aptitude  test  battery  was  administered  over  400, 000  times  between  1981  and  1992.  The  reliability,  validity, 
and  technical  fairness  of  the  test  battery  are  evaluated  in  this  paper.  Available  data  indicated  that  the  tests  comprising 
the  battery  had  acceptable  reliability.  A  composite  score  on  the  test  battery  predicted  performance  in  initial  ATCS 
qualification  training  at  the  FAA  Academy,  but  did  not  predict  outcomes  in  field  training.  Evidence  for  differential 
prediction  of  FAA  Academy  outcomes  on  the  basis  of  race  and  gender  was found  in  the  technical fairness  analyses.  However, 
the  validity  evidence  supported  using  the  battery  as  a  mass  testing  tool  by  the  FAA  in  rebuilding  the  controller  workforce. 


The  Air  Traffic  Control  Specialist  (ATCS)  occu¬ 
pation  is  the  single  largest  (about  17,000)  and  most 
publicly  visible  occupational  group  in  the  Federal 
Aviation  Administration  (FAA).  Air  traffic  control¬ 
lers  are  at  the  heart  of  a  web  of  radars,  computers,  and 
communication  facilities  that  comprise  an  increas¬ 
ingly  complex  and  busy  air  transportation  system. 
Appropriate  selection  of  personnel  into  these  critical 
positions  is  an  important  human  factors  problem. 
From  1981  through  1992,  the  FAA  used  noncom¬ 
pensatory,  multiple  hurdles  to  select  people  into  the 
ATCS  occupation.  The  first  hurdle  was  a  written 
aptitude  test  battery  administered  by  the  U.S.  Office 
of  Personnel  Management  (OPM;  Aul,  1991,  this 
volume).  Between  1981  and  1992,  the  test  was  ad¬ 
ministered  over  400,000  times.  Only  25,277  appli¬ 
cants  were  selected  on  the  basis  of  their  test  scores  to 
attend  the  FAA  Academy  initial  controller  training 
program,  the  second  major  hurdle  in  the  FAA’s 
selection  process.  The  purpose  of  this  paper  is  to 
describe  the  test  battery  and  evaluate  its  psychometric 
characteristics. 

Description  of  the  ATCS  Written  Test  Battery 

The  aptitude  test  battery  used  from  1981  to  1992 
was  developed  in  response  to  the  inadequacies  of  a 
more  traditional  civil  service  battery  that  was  in  use 
from  1964  to  1981.  The  effectiveness  of  the  1964 
civil  service  battery  was  questioned  as  early  as  1970, 


in  the  final  report  of  the  Air  Traffic  Controller  Career 
Committee  (Corson,  Berhard,  Catterson,  Fleming, 
Lewis,  Mitchell,  &  Ruttenberg,  1970).  A  review  of 
controller  training  in  the  mid-1970s  concluded  that 
improved  selection  methods  could  have  substantial 
monetary  implications  for  the  agency  (Henry, 
Ramrass,  Orlansky,  Rowan,  String,  &  Reichenbach, 
1975).  A  subsequent  technical  study  of  controller 
selection  tests  found  that  the  1 964  test  battery  was,  at 
best,  marginal  in  predicting  the  job  performance  of 
controllers  (Milne  &  Colmen,  1977).  Cooperative 
research  by  scientists  from  the  FAA  Office  of  Avia¬ 
tion  Medicine  (Dailey  &  Pickrel,  1984a,  b).  Civil 
Aeromedical  Institute  (CAMI;  Collins,  Boone,  & 
VanDeventer,  1 984),  and  OPM  (Rock,  Dailey,  Ozur, 
Boone,  &  Pickrel,  1982)  resulted  in  the  develop¬ 
ment,  validation,  and  implementation  of  a  new  writ¬ 
ten  ATCS  aptitude  test  battery  in  1 98 1  following  the 
strike  by  members  of  the  Professional  Air  Traffic 
Controller  Organization  (PATCO). 

Three  tests  comprised  the  new  1981  test  battery: 
the  Multiplex  Controller  Aptitude  Test  (MCAT), 
the  Abstract  Reasoning  Test  (ABSR),  and  the  Occu¬ 
pational  Knowledge  Test  (OKT).  The  MCAT  was  a 
timed,  paper-and-pencil  test  simulating  activities  re¬ 
quired  for  control  of  air  traffic.  Aircraft  locations  and 
direction  of  flight  were  indicated  with  graphic  sym¬ 
bols  on  a  simplified,  simulated  radar  display  (Figure 
2-1);  an  accompanying  table  provided  relevant 
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information  required  to  answer  the  item,  including 
aircraft  altitudes,  speeds,  and  planned  routes  of  flight. 
MCAT  test  items  required  examinees  to  identify 
situations  resulting  in  conflicts  between  aircraft,  to 
solve  time,  speed,  and  distance  problems,  and  to 
interpret  the  tabular  and  graphical  information.  The 
ABSR  was  a  50-item  civil  service  examination  (OPM- 
157).  To  solve  an  item,  examinees  determined  what 
relationships  existed  within  sets  of  symbols  or  letters. 
The  examinee  then  identified  the  next  symbol  or 
letter  in  the  progression  or  the  element  missing  from 
the  set.  A  sample  ABSR  item  is  presented  in  Figure2- 
2.  The  OKT  was  an  80-item  job  knowledge  test  that 
contained  items  related  to  seven  knowledge  domains 
generally  relevant  to  aviation,  and  specifically  rel¬ 
evant  to  air  traffic  control  phraseology  and  proce¬ 
dures.  The  OKT  was  developed  as  an  alternative  to 
self-reports  of  aviation  and  air  traffic  control  experi¬ 
ence.  The  OKT  was  found  to  be  more  predictive  of 
performance  in  ATCS  training  than  self-reports 
(Dailey  &  Pickrel,  1984b;  Lewis,  1978). 

The  scoring  of  the  MCAT,  ABSR,  and  OKT  is 
presented  in  Table  2-1.  The  sum  of  the  weighted 
MCAT  raw  score  and  the  ABSR  score  was  then 
transformed  into  a  linear  composite  ranging  from 
19.5  to  100.  This  “Transmuted  Composite”  (TMC) 
score  was  used  to  determine  employment  eligibility. 
For  applicants  without  specialized  prior  experience, 
education,  or  superior  academic  achievement,  a  mini¬ 
mum  TMC  of  75.1  was  required  to  qualify  at  the 
entry  level  GS-7  grade  (Aul,  1991,  this  volume).  If  an 
applicant  achieved  the  minimum  TMC,  extra  points 
were  awarded  on  the  basis  of  scores  on  the  OKT,  as 
noted  in  Table  2-1.  The  final  civil  service  rating 
(rating)  was  the  sum  of  TMC,  extra  points  on  the 
basis  of  OKT  scores,  and  any  adjudicated  veteran’s 
preference  points.  Ranking,  referral,  and  selection 
were  based  on  an  applicant’s  final  rating.  In  general, 
only  those  competitive  applicants  with  a  RATING  of  at 
least  90  were  selected  by  the  FAA  for  employment  as 
controllers.  The  distributions  of  applicant  and  selectee 
RATING  scores  are  presented  in  Figure  2-3,  based  on  over 
200,000  applicant  records  archived  at  CAMI  for  re¬ 
search  purposes. 

Psychometric  Characteristics 
Reliability 

Reliability  in  testing  refers  to  the  degree  to  which 
scores  on  a  test  are  free  from  errors  of  measurement 
(American  Educational  Research  Association,  Ameri¬ 


can  Psychological  Association,  &  National  Council 
on  Measurement  in  Education,  1985).  Test-retest 
reliability  is  an  estimate  of  the  degree  to  which  a 
person  will  obtain  approximately  the  same  score 
when  re-tested  after  some  time  interval.  The  test- 
retest  correlation  for  the  MCAT  was  estimated  at  .60 
in  a  sample  of  617  newly-hired  controllers  (Rock, 
Dailey,  Ozur,  Boone,  &  Pickrel,  1981,  p.  59).  Paral¬ 
lel  form  reliability  estimates  the  degree  to  which  an 
applicant  will  obtain  approximately  the  same  score 
on  a  different  version  of  the  same  test.  The  parallel 
form  reliability,  as  computed  on  the  same  sample, 
ranged  from  .42  to  .89  for  various  combinations  of 
items  (Rock  et  al.,  p.  103).  Internal  consistency 
estimates  the  degree  to  which  the  items  in  a  test  are 
homogenous  (Ghiselli,  Campbell,  &  Zedeck,  1981). 
Lilienthal  and  Pettyjohn  (1981)  examined  internal 
consistency  and  item  difficulties  for  ten  versions  of 
the  MCAT.  Cronbach’s  alpha  for  the  ten  versions 
ranged  from  .63  to  .93;  the  alphas  for  7  of  the  10 
versions  were  greater  than  .80.  The  available  data 
suggest  that  the  MCAT  had  acceptable  reliability  but 
was  vulnerable  to  practice  effects.  In  contrast,  no 
item  analyses,  parallel  form,  test-retest,  or  internal 
consistency  estimates  of  the  ABSR  (OPM-157)  test 
have  been  reported  by  the  FAA.  Therefore,  no  con¬ 
clusion  can  be  drawn  about  the  measurement  proper¬ 
ties  of  the  ABSR.  Published  data  are  available  for  the 
OKT.  Parallel  form  reliability  for  the  OKT  ranged 
from  .88  to  .91  (Rocketal.,  p.  65,  70).  The  internal 
consistency  estimate  of  reliability  (Kuder-Richardson 
Formula  20  (KR-20);  Kuder  &  Richardson,  1937) 
for  a  100-item  version  of  the  OKT  was  .95  (Rock  et 
al.,  p.  51).  Lilienthal  and  Pettyjohn  reported  Cronbach 
alphas  for  ten  versions  of  the  OKT  ranging  from  .85 
to  .94  on  a  sample  of  about  2,000  FAA  Academy  air 
traffic  control  students.  However,  test-retest  esti¬ 
mates  of  reliability  have  not  been  published.  The 
available  data  suggest  that  the  OKT  had  acceptable 
reliability. 

Validity 

Validity  generally  refers  to  the  appropriateness, 
meaningfulness,  and  usefulness  of  specific  inferences 
made  on  the  basis  of  test  scores  (American  Educa¬ 
tional  Research  Association,  American  Psychological 
Association,  &  National  Council  on  Measurement  in 
Education,  1985).  Validity  in  the  specific  context  of 
employee  selection,  as  discussed  in  the  federal  Uni¬ 
form  Guidelines  on  Employee  Selection  Procedures 
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(Equal  Employment  Opportunity  Commission, 
1978),  refers  to  the  degree  to  which  test  scores  used 
for  making  employment  decisions  are  predictive  or 
correlated  with  important  and/or  critical  work  out¬ 
comes,  elements,  or  behaviors.  The  ATCS  aptitude 
test  battery  has  been  validated  against  two  classes  of 
work  outcomes  in  previous  studies:  (a)  performance 
in  the  FAA  Academy  initial  training  programs  (screen; 
Della  Rocco,  this  volume);  and  (b)  outcomes  of  on- 
the-job  field  training  (OJT)  at  the  first  assigned  field 
facility  (STATUS;  Manning,  this  volume). 

As  the  final  civil  service  rating  was  the  basis  of 
operational  personnel  selection  decisions,  this  retro¬ 
spective  analysis  examined  the  validity  of  the  final 
RATING  as  a  predictor  of  these  two  criteria  for  the 
15,876  controllers  who  survived  the  FAA  Academy 
and  were  placed  into  field  training  between  1981  and 
1992.  The  field  training  status  criterion  was  coded 
as  follows:  1  =  Reached  Full  Performance  Level\  2  = 
Still  in  Training  (Developmental)',  3  =  Switched  facili¬ 
ties',  4  =  Switched  options',  and  5  =  Failed.  As  recom¬ 
mended  by  Manning,  persons  who  had  attrited  for 
reasons  other  than  performance  were  excluded  from 
the  validity  analysis.  Because  the  samples  had  been 
truncated  by  selection  first,  on  rating,  and  second, 
by  selection  on  FAA  Academy  score  (screen),  correc¬ 
tions  for  restriction  in  range  were  made  as  shown  in 
Table  2-2,  using  the  formulae  presented  by  Ghiselli, 
Zedeck,  and  Campbell  (1984).  Correlations  between 
civil  service  rating  and  FAA  Academy  screen  scores 
were  corrected  for  direct  restriction  in  range  due  to 
explicit  selection  on  rating.  The  corrections  were 
based  on  the  population  standard  deviations  pre¬ 
sented  in  the  Tables  2-3  through  2-6.  Correlations 
between  FAA  Academy  (screen)  and  field  training 
outcomes  (status)  were  corrected  for  direct  restric¬ 
tion  in  range  due  to  explicit  selection  on  the  FAA 
Academy  score.  Finally,  correlations  between  civil 
service  RATING  and  field  training  outcomes  were  cor¬ 
rected  for  incidental  restriction  in  range  due  to  selec¬ 
tion  of  the  samples  on  FAA  Academy  SCREEN  scores. 
The  uncorrected,  zero-order  correlations  are  pre¬ 
sented  in  the  lower  left  corner  of  each  correlation 
matrix  and  the  corrected  correlations  in  the  upper 
right  corner.  The  corrected  correlation  matrices  for 
each  sample  were  submitted  to  regression  analysis  to 
estimate  the  validity  of  rating  as  a  predictor  of  FAA 
Academy  screen  and  field  training  STATUS. 


In  view  of  the  iterations  of  the  FAA  Academy 
initial  training  programs,  described  by  Della  Rocco, 
and  differences  in  field  training  as  described  by 
Manning,  the  sample  was  divided  into  four  groups: 
(a)  1982-85  FAA  Academy  Terminal  program  gradu¬ 
ates  assigned  to  training  in  the  terminal  option  (Table 
2-3);  (b)  1982-85  FAA  Academy  En  route  program 
graduates  assigned  to  field  training  in  the  en  route 
option  (Table  2-4);  (c)  1986-92  FAA  Academy  Non¬ 
radar  Screen  assigned  to  field  training  in  the  terminal 
option  (Table  2-5);  and  (d)  1986-92  FAA  Academy 
Non-radar  Screen  graduates  assigned  to  field  training 
in  the  en  route  option  (Table  2-6). 

On  one  hand,  this  retrospective  analysis  found 
that  the  final  civil  service  rating  had  acceptable 
validity  as  a  predictor  of  performance  in  initial  train¬ 
ing  at  the  FAA  Academy  for  all  four  groups.  The 
uncorrected  correlations  between  rating  and  screen 
were  statistically  significant  and  ranged  from  .  1 78  to 
.222  across  the  four  groups.  The  corrected  correla¬ 
tions  between  civil  service  RATING  and  FAA  Academy 
SCREEN  ranged  from  .458  to  .502.  On  the  other  hand, 
regression  analyses  of  the  corrected  correlation  matri¬ 
ces  found  that  RATING  was  a  relatively  poor  predictor 
of  field  training  STATUS  in  three  of  the  four  FAA 
Academy  and  field  training  variations  (Table  2-7). 
The  standardized  regression  weight  (P)  for  rating 
was  not  statistically  significant  for  graduates  of  1 982- 
85  Terminal  Screen  program  assigned  to  field  train¬ 
ing  in  the  terminal  option  (P  =  -.025,  ns)  or  for 
1986-92  Non- radar  Screen  program  graduates  as¬ 
signed  to  either  terminal  (P  =  -.041 ,  ns)  or  en  route  (P 
=  -.010,  ns)  field  training.  However,  RATING  did  enter 
into  the  regression  equation  predicting  STATUS  in  en 
route  field  training,  with  a  standardized  regression 
weight  of -.107  (/?<. 001),  for  graduates  ofthe  1982-85 
En  Route  Screen  program,  as  shown  in  Table  2-7. 

Fairness 

The  FAA,  in  its  1993  Diversity  Plan,  made  a 
commitment  to  attract,  retain,  develop,  and  manage 
a  diverse  work  force  that  visibly  reflected  the  Ameri¬ 
can  population  at  large  by  the  year  2000.  Achieving 
this  goal  will  require  substantial  changes  in  the  demo¬ 
graphic  profile  of  the  ATCS  occupation.  Air  traffic 
control  is  a  career  field  in  which  female  and  minority 
workers  have  been  historically  under-represented  rela¬ 
tive  to  the  American  population  at  large.  The  techni- 
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cal  fairness  of  the  written  ATCS  aptitude  test  battery 
was  recently  examined  in  a  series  of  papers  as  the  first 
step  toward  assessing  to  what  degree,  if  any,  the 
battery  may  have  served  as  an  “engine  of  exclusion” 
(Seymour,  1988)  of  women  and  minorities  from  the 
ATCS  occupation  (Young,  Broach,  &  Farmer,  1996; 
Broach,  Farmer,  &  Young,  1997).  Technical  fairness 
refers  to  the  regression  model  of  test  bias  for  which 
there  is  a  reasonable  professional  consensus,  as  em¬ 
bodied  in  the  1985  Standards  for  Educational  and 
Psychological  (American  Educational  Research 

Association,  American  Psychological  Association,  & 
National  Council  on  Measurement  in  Education), 
rather  than  a  socially  constructed  standard  regarding 
test  use  (Sackett  &  Wilk,  1994;  Gottfredson,  1994). 
Technical  fairness  in  the  regression  model,  and  under 
the  Uniform  Guidelines  on  Employee  Selection  Proce¬ 
dures  (Equal  Employment  Opportunity  Commis¬ 
sion,  1978),  encompasses  two  issues.  First,  the  effect 
on  protected  groups  arising  from  use  of  a  particular 
cut  score  on  the  predictor  must  be  evaluated.  A 
selection  rate  for  any  protected  group  that  is  less  than 
four-fifths  (4/ 5  or  80%)  of  that  of  the  majority  group 
will  “...  generally  be  regarded  by  the  Federal  enforce¬ 
ment  agencies  as  evidence  of  adverse  impact”  (29 
CFR  1 607.4.  D).  Second,  where  the  use  of  a  selection 
procedure  results  in  adverse  impact,  the  Uniform 
Guidelines  on  Employee  Selection  Procedures  require 
that  the  test  user  evaluate  the  degree  to  which  differ¬ 
ential  predictions  of  future  job  performance  are  made 
from  selection  test  scores  by  subgroup  (29  CFR 
l607.l4.B.(8).(b)). 

Gender 

Previous  research  on  written  ATCS  selection  tests 
suggested  that  mean  score  differences  by  gender  were 
insignificant  (Rock,  Dailey,  Ozur,  Boone,  &  Pickrel, 
1984a,  pp.  476)  and  that,  overall,  “the  evidence  for 
adverse  impact  against  women  based,  on  this  sample, 
was  marginal,  at  best”  (Rock,  Dailey,  Ozur,  Boone, 
&  Pickrel,  1984b,  pp.  507).  This  conclusion  was 
based  on  results  of  a  1 984  study  in  which  57%  of  men 
(w  =  3,835)  passed  the  written  test,  compared  to  45% 
of  women  (w  =  1,473).  The  adverse  impact  ratio  in 
this  case  was  0.78,  rather  than  the  0.80  required 
under  the  “four-fifths  rule  of  thumb.”  Young,  Broach 
and  Farmer  (1996)  analyzed  archival  records  for 
170,578  applicants.  Based  on  test  scores,  OPM  de¬ 
termined  the  eligibility  of  applicants  for  employment 
by  the  FAA.  OPM  codes  indicating  that  an  applicant 


had  either  failed  the  test  CIA’)  or  scored  too  low  for 
consideration  (‘IS’)  were  categorized  as  test  failures. 
All  other  ineligibility  codes  were  categorized  as  “other 
ineligible,”  and  excluded  from  the  analysis.  OPM 
codes  indicating  eligibility  were  categorized  as  “eli¬ 
gible”  for  employment.  The  adverse  impact  analysis 
compared  the  proportion  of  women  considered  eli¬ 
gible  with  that  of  men.  The  analysis  is  presented  in 
T  able  2-8 .  A  significantly  lower  percentage  of  women 
(42.0%)  than  men  (58.1%)  were  coded  by  OPM  as 
eligible  for  employment  as  controllers,  based  on  test 
scores  {Z  =  -3.58,^  <  .001).  The  ratio  of  female  to 
male  selection  rates  was  .72.  Using  the  4/5ths  rule  of 
thumb  of  the  Uniform  Guidelines  on  Employee  Selec¬ 
tion  Procedures^  it  appeared  that  the  use  of  scores  on 
the  written  ATCS  aptitude  test  battery  to  determine 
eligibility  for  employment  in  the  ATCS  occupation 
between  1981  and  1992  resulted  in  statistically  signifi¬ 
cant  adverse  impact  on  female  applicants. 

Given  the  finding  that  there  appeared'to  be  adverse 
impact  against  women,  the  Uniform  Guidelines  on  Em¬ 
ployee  Selection  Procedures  (29  CFR  1 607. 1 4. B . (8) .  (b)) 
and  Standards  for  Educational  and  Psychological  Test¬ 
ing  (Standard  1 .20,  p.  17)  next  required  an  investiga¬ 
tion  of  the  relationship  between  test  scores  and  job 
performance  for  evidence  of  differential  prediction 
by  subgroup.  The  classical,  regression-based  model 
of  test  bias  was  used  as  the  analytic  framework  by 
Young,  Broach,  and  Farmer  to  evaluate  the  degree  to 
which  the  written  ATCS  test  battery  differentially 
predicted  performance  in  initial  ATCS  qualification 
training  at  the  FAA  Academy.  After  correcting  corre¬ 
lations  between  test  score,  FAA  Academy  composite 
score,  and  gender  for  explicit  and  implicit  restric¬ 
tions  in  range  due  to  prior  selection  of  the  sample  on 
aptitude  test  score,  the  null  hypothesis  of  a  common 
regression  line  for  the  genders  was  rejected,  suggest¬ 
ing  the  presence  of  some  degree  of  test  bias.  Statisti¬ 
cally  significant  differences  by  gender  in  regression 
slopes  and  intercepts  were  found  in  the  step-down 
regression  analysis,  indicating  the  need  for  separate 
regression  equations  for  men  and  women.  The  regres¬ 
sion  line  for  men  slightly  over-predicted  the  perfor¬ 
mance  of  women  in  the  FAA  Academy,  as  shown  in 
Figure  2-4.  The  practical  consequence  of  the  appar¬ 
ent  differential  prediction  was  that  women  effectively 
needed  a  higher  test  score  than  men  to  have  an  equal 
likelihood  of  passing  the  initial  qualification  training 
at  the  FAA  Academy. 
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Race 

A  formal  adverse  impact  analysis  on  the  basis  of  race 
was  not  technically  feasible,  as  racial  identifiers  were  not 
collected  for  ATCS  job  candidates.  However,  previous 
research  (Rock,  Dailey,  Ozur,  Boone,  &  Pickrel,  1984) 
found  that  the  OPM  test  battery  had  adverse  impact 
against  African-Americans  and  Hispanics.  In  view  of 
the  previous  research,  Broach,  Farmer,  and  Young  ( 1 997) 
investigated  the  relationship  between  test  scores  and 
performance  in  the  FAA  Academy  for  evidence  of 
differential  prediction  on  the  basis  of  race.  The  classical, 
regression-based  model  of  test  bias  was  again  used  as  the 
analytic  framework  to  evaluate  the  degree  to  which  the 
written  ATCS  test  battery  differentially  predicted  per¬ 
formance  in  initial  ATCS  qualification  training  at  the 
FAA  Academy  for  African-American  and  white  control¬ 
lers.  After  correcting  correlations  between  test  score, 
FAA  Academy  composite  score,  and  race  for  explicit  and 
implicit  restrictions  in  range,  the  null  hypothesis  of  a 
common  regression  line  for  African-American  and  white 
controllers  was  rejected,  suggesting  the  presence  of  some 
degree  of  test  bias.  Statistically  significant  differences  by 
race  in  regression  slopes  and  intercepts  were  found  in  the 
step-down  regression  analysis,  indicating  the  need  for 
separate  regression  equations  for  African-American  and 
white  controllers.  The  regression  line  for  whites  over- 
predicted  the  performance  of  African-Americans  in  the 
FAA  Academy.  The  practical  consequence  of  the  appar¬ 
ent  differential  prediction  was  that  African-American 
controllers  effectively  needed  a  higher  test  score  than 
white  controllers  to  have  an  equal  likelihood  of  passing 
the  initial  qualification  training  at  the  FAA  Academy,  as 
shown  in  Figure  2-5. 

Discussion 

The  written  ATCS  aptitude  test  battery  in  use  from 
1981  through  1992  was  developed  to  enable  the  agency 
to  make  reasonable  predictions,  for  large  numbers  of 
applicants,  about  their  future  job  performance  as  the 
basis  for  initial  selection  into  a  safety-related  occupa¬ 
tion.  From  a  practical  perspective,  the  primary  function 
of  the  written  ATCS  test  battery  was  to  winnow  a  huge 
pool  of  applicants  down  to  a  smaller,  manageable  num¬ 
ber  suitable  for  further  intensive,  expensive  evaluation. 
On  one  hand,  the  written  ATCS  aptitude  test  battery,  as 
used  from  1981  to  1992,  achieved  that  organizational 
goal.  The  tests  comprising  the  battery  produced  reliable 
scores.  A  composite  of  those  reliable  scores,  used  as  the 
basis  for  selection  decisions,  was  valid  as  a  predictor  of 
near-term  (relative  to  date-of-hire)  training  outcomes. 


On  the  other  hand,  the  composite  score  was  not  a 
valid  predictor  of  far-term  (2  to  3  years  from  hire) 
field  training  outcomes.  Moreover,  the  fairness  analy¬ 
ses  suggested  that  the  battery  had  adverse  impact 
against  women,  and  was  likely  to  have  had  adverse 
impact  on  African-American  job  candidates.  The  test 
battery  over-predicted  performance  in  initial  ATCS 
qualification  training  at  the  FAA  Academy  on  the 
basis  of  gender  and  race.  Adverse  impact  and  over¬ 
prediction  of  training  and  job  performance  is  com¬ 
monly  reported  for  tests  of  cognitive  ability  similar  to 
the  ATCS  test  battery,  such  as  the  General  Aptitude 
Test  Battery  (GATB)  (Hartigan  &  Wigdor,  1989; 
Schmidt,  1988;  Wigdor  &  Sackett,  1993).  This  out¬ 
come  does  not  result  because  scores  on  the  ATCS 
aptitude  test  battery  mean  something  different  for 
women  and  African  Americans.  The  test  is  not  biased 
in  that  sense.  Rather,  the  adverse  impact  and  over¬ 
prediction  of  subsequent  performance  in  initial  train¬ 
ing  for  both  groups  results  from  the  interplay  of  two 
factors:  the  lower  average  scores  for  African  Ameri¬ 
cans  and  women  relative  to  white  males,  and  the  less- 
than-perfect  validity  of  the  composite  test  score.  As  a 
consequence  of  these  facts,  the  predicted  perfor¬ 
mance  of  minorities  and  women  is  higher  than  their 
actual  performance.  The  practical  result  is  that  more 
women  and  minorities  were  hired  on  the  basis  of  their 
predicted  performance  than  would  have  been  hired 
on  the  basis  of  their  actual  job  performance. 

In  conclusion,  the  written  test  battery  was  a  valid 
mass  testing  tool,  with  which  the  FAA  could  reduce 
the  applicant  pool  to  a  smaller,  affordable  number  for 
more  intensive  evaluation  of  their  aptitude  for  the 
controller  occupation.  The  relative  costs  of  assess¬ 
ment  methods  must  be  carefully  considered,  particu¬ 
larly  in  view  of  the  substantial  numbers  of  job 
applicants.  For  example,  the  cost  of  administering 
the  paper-and-pencil  ATCS  test  battery  was  about 
$20  per  examinee,  compared  with  a  cost  of  about 
$10,000  per  person  for  initial  training  at  the  FAA 
Academy  (Broach  &  Brecht-Clark,  1993).  There  was 
significant  financial  utility  for  the  agency  in  using  a 
valid  test  as  the  first  step  in  a  multiple-hurdle,  se¬ 
quential  personnel  selection  system.  Moreover,  the 
test  battery  was  fair,  in  that  test  scores  over-predicted 
subsequent  training  performance  of  African  Ameri¬ 
cans  and  women.  Overall,  the  written  AT CS  aptitude 
test  battery  was  an  inexpensive,  practical,  valid,  fair, 
and  invaluable  tool  in  rebuilding  the  controller 
workforce  between  1981  and  1992. 
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CRAFT 

ALTITUDe 

SPEED 

ROUTE 

10 

7000 

480 

AGKHC 

20 

7000 

480 

BOJE 

30 

7000 

240 

AGJE 

40 

6500 

240 

CHKJF 

50 

6500 

240 

DIKGB 

60 

8000 

4B0 

DIKJE 

70 

8000 

480 

FJKID 

S<MWPLE<yiESTION 


WHICH  AIRCRAFT  WILL  CONFLICT? 

A.  60  AND  70 

B.  40  AND  70 

C.  20  AND  30 

0.  NONE  OF  THESE 


N 


MILEAGE  SCALE 


Figure  2-1:  Example  Multiplex  Controller  Aptitude  Test  (MCAT)  item 


Symbols 


IS]  ^ 

sO  ? 

@  A  P]  ^ 

ABC  D  E 

V  A  ? 

^  I'  ^  y  \ 

A  B  C  D  E 


Letters 

1)  XCXDXEX  a)FX  b)FG  c)XFd)EF  e)XG 

2)  ARCSETG  A)  HI  b)HUC)UJ  d)UI  e)IV 

Figure  2-2:  Example  Abstract  Reasoning  (ABSR)  test  item 
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70  75  80  85  90  95  100 

Test  score  (TMC) 


Figure  2-5:  Regression  of  test  score  (TMC)  on  FAA  Academy  score  (SCREEN)  by  race 


Table  2-1:  ATCS  Aptitude  test  battery  scoring 


Test 

OPM# 

Scoring 

Weight 

N  Items 

MCAT 

510 

N  Right 

2 

110 

ABSR 

157 

N  Right  -  (0.25*N  Wrong) 

1 

50 

OKT 

512 

N  Right 

a 

80 

Note:  Extra  points  awarded  as  follows  for  OKT  raw  scores:  0-51  =  0  extra  points;  52-55  =  3  extra  points; 
56-59  =  5  extra  points;  60-63  =  1 0  extra  points;  64-80  =  1 5  extra  points  for  computation  of  the 
civil  service  rating. 


Table  2-2:  Correlation  matrix  structure  for  regression  analyses 


RATING 

SCREEN 

STATUS 

RATING 

F  0 

'^(RATING) 

^SCREEN) 

SCREEN 

rs 

l^e 

^(SCREEN) 

STATUS 

Note:  Sample  correlation  matrix  structure  shown  below  the 

diagonal,  corrected  matrix  structure  above  the  diagonal, 
r  =  sample  correlation;  Tp  =  correlation  corrected 

^  *  ^(RATING) 

for  explicit  selection  on  RATING;  r;  =  correlation 

‘  ‘(SCREEN) 

corrected  for  incidental  restriction  in  range  due  to 
selection  on  SCREEN;  and  Tp  =  correlation 

^(SCREEN) 

corrected  for  explicit  selection  on  SCREEN 
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Table  2-3:  Descriptive  statistics  and  correlations  for  1982-85  Terminal  Screen  graduates 

assigned  to  Terminal  OJT 


Variable 

Sample’ 

Population^" 

,C 

Correlations*^ 

N 

M 

SD 

N 

M 

SD 

(1)  (2)  (3) 

(1)  RATING 

2,814 

92.31 

5.72 

99,887 

74.93 

14.24 

.493  -.168 

(2)  SCREEN 

3,174 

79.45 

5.48 

4,686 

73.82 

11.32 

.222***  -.303 

(3)  STATUS 

3,174 

88.0% 

FPL 

-.037  -.146*** 

12.0% 

NOT  FPL 

Notes:  ^STATUS  outcomes  for  first  assigned  field  facility  only  as  of  March  1997  **  p  <  .01,  ***p  <  .001 

‘'RATING  based  on  CAMI  records  for  examinees  tested  1982-85 
‘'SCREEN  based  on  Terminal  Screen  entrants  who  did  not  withdraw,  1982-1985 
‘‘Uncorrected  correlations  in  lower  left  half,  corrected  in  upper  right  half. 


Table  2-4:  Descriptive  statistics  and  correlations  for  1982-85  En  Route  Screen  graduates 

assigned  to  En  Route  OJT 


Variable 

Sample’ 

Population‘'' 

c 

Correlations** 

N 

M 

SD 

N 

M 

SD 

(1) 

(2)  (3) 

(1)  RATING 

4,021 

92.56 

5.37 

99,887 

74.93 

14.24 

.502  -.256 

(2)  SCREEN 

4,276 

82.02 

7.17 

4,686 

73.82 

11.32 

-.351 

(3)  STATUS 

4,276 

66.1%  FPL 

-.047** 

-.231** 

33.9%  NOT  FPL 

Notes:  ^STATUS  outcomes  for  first  assigned  field  facility  only  as  of  March  1 997  **  p  <  .01 ,  ***p  <  .001 

‘'RATING  based  on  CAMI  records  for  examinees  tested  1982-85 
‘'SCREEN  based  on  En  route  Screen  entrants  who  did  not  withdraw,  1 982- 
1985 

‘‘Uncorrected  correlations  in  lower  left  half,  corrected  in  upper  right  half. 


Table  2-5:  Descriptive  statistics  and  correlations  for  1986-92  Non-radar  Screen  graduates 

assigned  to  Terminal  OJT 


Variable 

Sample" 

Population‘ 

),C 

Correlations** 

N 

M 

SD 

N 

M 

SD 

(1)  (2)  (3) 

(1)  RATING 

2,765 

93.31 

5.15 

106,201 

76.33 

14.67 

.458  -.095 

(2)  SCREEN 

3,145 

78.28 

6.07 

12,756 

72.26 

11.80 

.178***  -.137 

(3)  STATUS 

3,298 

95.0%  FPL 

-.027  -.071*** 

5.0%  NOT  FPL 

Notes:  "STATUS  outcomes  for  first  assigned  field  facility  only  as  of  March  1 997  **  p  <  .01 ,  ***p  <  .001 

‘'RATING  based  on  CAMI  records  for  examinees  tested  1982-85 
‘'SCREEN  based  on  Non-radar  Screen  entrants  who  did  not  withdraw, 

1982-1985 

‘‘Uncorrected  correlations  in  lower  left  half,  corrected  in  upper  right  half. 
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Table  2-6:  Descriptive  statistics  and  correlations  for  1986-92  Non-radar  Screen  graduates 

assigned  to  En  Route  OJT 


Variable 

Sample" 

Population' 

>,c 

Correlations'" 

N 

M 

SD 

N 

M 

SD 

(1) 

(2) 

(3) 

(1)  RATING 

3,969 

93.79 

5.06 

106,201 

76.33 

14.67 

.493 

-.129 

(2)  SCREEN 

4,449 

80.26 

5.89 

12,756 

72.26 

11.80 

1 02*** 

-.247 

(3)  STATUS 

4,732 

77.3%  FPL 

-.014 

.126*** 

22.7%  NOT  FPL 

Notes: ’STATUS  outcomes  for  first  assigned  field  facility  only  as  of  March  1 997  **  p  <  .01,  ***p  <  .001 

'’RATING  based  on  CAMI  records  for  examinees  tested  1 982-85 
'SCREEN  based  on  Non-radar  Screen  entrants  who  did  not  withdraw,  1 982-1 985 
'‘Uncorrected  correlations  in  lower  left  half,  corrected  in  upper  right  half. 


Table  2-7:  Civil  service  RATING  validity  in  predicting  FIELD  TRAINING  outcome  by  sample, 

based  on  corrected  correlations 


Sample 

R 

P-RATING 

P-SCREEN 

Period 

FAA  Academy 

Field  OJT 

1 982-85 

Terminal  Screen 

Terminal 

.304*** 

-.025 

-  291  *** 

1982-85 

En  Route  Screen 

En  route 

.363*** 

-  1 07*** 

-  297*** 

1986-92 

Non-radar  Screen 

Terminal 

142*** 

-.041 

_  118*** 

1 986-92 

Non-radar  Screen 

En  route 

247*** 

-.010 

-  242*** 

*p  <  .05,  **p  <  .01 ,  ***p  <  .00 


Table  2-8:  Adverse  impact  analysis  by  gender  based  on  OPM  eligibility  codes 


OPM  Eligibility 

Males 

Sex 

Females 

Row  totals 

Eligible 

69,056 

(58.1%) 

14,564 

(42.0%) 

84,070 

Failed  test 

49,902 
(41 .9%) 

20,077 

(58.0%) 

69,979 

Column  totals 

1 1 8,985 

34,641 

53,599 

Notes:  Percents  are  column  percentages 
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CHAPTERS: 

FAA  ACADEMY  AIR  TRAFFIC  CONTROL  SPECIALIST 
SCREENING  PROGRAMS  AND  STRIKE  RECOVERY 

Pamela  Della  Rocco 

FAA  Civil  Aeromedical  Institute,  Oklahoma  City,  Oklahoma 


In  1976,  the  Federal  Aviation  Administration  (FAA)  implemented  screening  programs  for  Air  Traffic  Control 
Specialists  (ATCSs)  at  the  FAA  Academy  which,  in  combination  with  a paper-and^pencil selection  test,  provided  a 
robust  two-stage  selection  system  used  in  recovery  from  the  1981  Professional  Air  Trajftc  Controller  Organization 
(PA  TCO)  strike.  The  purpose  of  the  present  paper  is  to  review  data  from  the  FAA  Academy  screening  programs  with 
regard  to  their  role  in  the  FAA  strike  recovery  ejforts,  as  well  as  the  evolution  of  the  programs  in  the  post-strike  years. 
Data  are  presented  for  pass  rates,  demographics,  and  evidence  of  validity  of  the  programs. 


At  the  time  of  the  1981  Professional  Air  Traffic 
Controller  Organization  (PATCO)  strike,  the  Fed- 
eral  Aviation  Administration  (FAA)  selected  Air  Traf¬ 
fic  Control  Specialists  (ATCSs)  through  a  two-stage 
process  (Della  Rocco,  Manning,  &  Wing,  1990). 
The  first  stage  was  the  Office  of  Personnel  Manage¬ 
ment  (OPM)  written  air  traffic  control  (ATC)  test 
battery  (Broach,  this  volume).  The  second  stage  was 
a  FAA  Academy  initial  qualification  course.  This 
two-stage  process  provided  a  robust  foundation  for 
recovery  of  the  controller  workforce  in  the  wake  of 
the  firing  of  the  striking  ATCSs.  The  purpose  of  the 
present  paper  is  to  review  the  FAA  Academy  screen¬ 
ing  programs  with  regard  to  their  role  in  the  FAA 
recovery  from  the  strike,  as  well  as  the  evolution  of 
the  programs  in  the  post-strike  years.  Data  are  pre¬ 
sented  for  pass  rates,  demographics,  and  validity  of 
the  programs. 

Background 

The  FAA  has  conducted  research  on  the  selection 
of  ATCSs  since  the  1950s  (Collins,  Boone,  & 
VanDeventer,  1984).  Written  aptitude  tests  for  the 
occupation  were  implemented  as  early  as  1964  and 
were  administered  nation-wide.  Despite  the  increased 
validity  of  the  written  Civil  Service  Commission 
(CSC)  selection  tests  over  the  previous  system  based 
on  prior  experience  alone,  the  high  occupational 
attrition  rates  were  a  concern  through  the  late  1960s 
and  into  the  early  1970s.  In  1974,  for  example,  the 


failure  rate  was  about  43%  for  en  route  trainees  and 
38%  for  terminal  trainees  (Henry,  Ramrass,  Orlansky, 
Rowan,  String,  &  Reichenbach,  1975).  This  training 
attrition  typically  occurred  two  to  three  years  into  an 
individual’s  career  (Manning,  1991).  During  the 
early  1970s,  several  groups  examined  the  FAA’s  air 
traffic  training  and  selection  process  (Corson,  1970; 
Henry  et  al.,  1975;  Committee  on  Government 
Operations,  1976).  The  reports  recommended  the 
development  of  a  validated,  standardized,  and  cen¬ 
tralized  pass/ fail  training  program,  designed  to  screen 
persons  who  lacked  sufficient  potential  to  become 
fully  certified  ATCSs  early  in  their  career  (Boone, 
1984).  The  FAA  committed  to  the  development  of 
screening  programs,  which  would  follow  successful 
performance  on  the  written  CSC  (renamed  Office  of 
Personnel  Management  in  1978)  air  traffic  test  bat¬ 
tery  (Boone,  1984). 

Description  of  FAA  Academy  Screening  programs 
Pre-strike  Screening  programs 

In  1 976,  the  pass/fail  FAA  Academy  initial  qualifica¬ 
tion  courses  were  implemented.  They  were  based  on 
non-radar  air  traffic  control  procedures.  Separate  courses 
existed  for  the  two  career  options,  En  Route  and  Termi¬ 
nal.  Entrants  were  assigned  to  an  option  based  upon 
agency  staffing  needs.  The  screening  programs  evolved 
from  training  previously  administered  by  the  FAA  Acad¬ 
emy.  The  courses  were  designed  to  assess  the  aptitude 
of  an  individual  with  no  prior  knowledge  of  the 
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occupation.  These  screening  courses  were  the  equiva¬ 
lent  of  a  miniaturized  training-testing  model  of  selec¬ 
tion  (Manning,  Kegg,  &  Collins,  1989;  Della  Rocco, 
Manning,  &  Wing,  1990). 

The  Terminal  and  En  Route  Screen  programs  con¬ 
tained  two  pass/fail  phases  covering  (1)  an  introduction 
to  aviation  and  air  traffic  control  and  (2)  procedures  for 
non-radar  control  of  air  traffic  in  the  airspace  relevant  to 
each  option  (Boone,  1984).  The  first  pass/fail  segment 
was  strictly  academic  and  covered  topics  such  as  prin¬ 
ciples  of  flight,  meteorology,  air  traffic  service  and  the 
National  Airspace  System.  However,  it  was  the  non¬ 
radar  procedures  phase  of  training  that  identified  the 
candidates  with  less  chance  of  succeeding  in  the  occupa¬ 
tion.  The  non-radar  phase  for  both  options  included 
academics,  laboratory  simulation  problems,  and  the 
Controller  Skills  Test  (CST).  During  the  academic 
classroom  training,  entrants  were  taught  the  rules  and 
principles  of  non-radar  ATC.  Students  received  a  score 
from  this  academic  training  which  was  included  in  the 
final  non-radar  phase  score.  During  the  laboratory 
training,  entrants  were  required  to  apply  the  rules  and 
principles  learned  in  the  classroom  to  air  traffic  simula¬ 
tion  problems.  Each  entrant  performed  non-radar  con¬ 
troller  job  tasks  in  one  or  two  thirty-minute  scripted 
scenarios  daily.  An  individual  instructor  was  assigned  to 
each  student  for  each  scenario.  Following  the  scenario, 
the  instructor  reviewed  the  student’s  performances  and 
provided  feedback  on  errors  as  well  as  correct  applica¬ 
tion  of  the  rules  and  scenario  scores.  Instructors  as¬ 
signed  a  technical  assessment  (number  of  errors),  as  well 
as  an  instructor  assessment  (subjective  evaluation  of 
student  performance)  for  each  scenario.  The  laboratory 
scenarios  were  structured  to  escalate  in  complexity.  Five 
of  six  of  the  final  laboratory  problems  were  graded  and 
counted  in  the  final  score.  The  CST  was  the  final  test 
administered  in  the  non-radar  phase.  It  was  a  timed, 
paper-and-pencil,  multiple-choice  assessment  of  the 
student’s  ability  to  apply  non-radar  ATC  rules  and 
procedures.  A  final  composite  score  of  at  least  70  (out  of 
a  possible  score  of  1 00)  was  required  to  pass  the  non¬ 
radar  phase.  Failure  to  achieve  a  composite  score  of  70 
resulted  in  removal  from  the  ATCS  job.  Although  the 
En  Route  and  Terminal  Screen  programs  were  modified 
over  the  years,  the  basic  structure  remained  much  the 
same  between  1976  and  1985. 

At  the  time  of  the  strike  in  1981,  the  En  route  and 
Terminal  Screen  programs  had  effectively  achieved  the 
goal  of  reducing  field  attrition  (Manning,  1991).  Be¬ 
tween  1971  and  1975,  field  attrition  in  both  options 


averaged  41%.  Following  the  implementation  of  the 
screening  programs  between  1976  and  1981,  FAA  Acad¬ 
emy  attrition  rates  were  29%  and  subsequent  field 
attrition  was  cut  to  8%.  Thus,  within  the  first  few 
months  of  appointment  to  the  occupation,  the  FAA 
Academy  programs  screened  out  most  of  those  indi¬ 
viduals  who  lacked  sufficient  potential  to  become  fully 
certified  ATCSs.  The  pass  rate  was  67%  for  En  Route 
Screen  entrants  and  74%  for  Terminal,  with  an  overall 
average  of  70%.  Of  historical  note,  subsequent  to  the 
strike,  a  new  Office  of  Personnel  Management  (OPM) 
test  battery  was  implemented,  replacing  the  previous 
CSC  battery  (Broach,  this  volume;  Collins,  Boone,  & 
VanDeventer,  1984). 

A  key  feature  of  the  1976  effort  was  to  implement  a 
research  database  system  to  document  and  monitor  the 
effectiveness  of  the  program.  The  database  was  estab¬ 
lished  at  the  Civil  Aeromedical  Institute  (CAMI).  Data 
were  collected  on  all  FAA  Academy  entrants.  Planning 
for  the  second-stage  pass/fail  screening  programs  was 
influenced  by  requirements  identified  by  publication  of 
the  original  Uniform  Guidelines  on  Employee  Selection 
Procedures  in  1966  (Boone,  1984).  The  database  pro¬ 
vided  a  source  for  monitoring  compliance  with  those 
guidelines.  In  1984,  the  CAMI  tracking  database  was 
established  to  track  FAA  Academy  graduates  through 
Full  Performance  Level  (FPL;  Manning,  this  volume). 
This  database  provided  criterion  measures  for  program 
validation. 

Strike  recover^",  1981-1985 

By  the  time  of  the  strike  in  1981,  the  FAA  had  five 
and  a  half  years  of  experience  using  the  En  Route  and 
Terminal  initial  qualification  courses  as  screening  pro¬ 
grams.  In  addition,  data  on  all  entrants  had  been  col¬ 
lected  in  the  research  database.  Thus,  the  system  was  in 
place  to  select  and  monitor  the  progress  of  ATCS 
candidates,  including  individuals  who  had  no  knowl¬ 
edge  of  the  occupation.  The  FAA  Academy  only  needed 
to  gear  up  to  manage  the  increased  student  load  result¬ 
ing  from  the  strike  recovery  effort.  Prior  to  the  strike, 
8,262  candidates  (4, 1 38  En  Route  and  4, 1 24  Terminal) 
entered  the  programs  in  the  approximately  five  and  a 
half  year  period  between  January  1 976  and  August  1981 
(Manning,  Kegg,  &  Collins,  1988).  Following  the 
strike,  13,533  candidates  entered  during  the  four-year 
period  between  August  1981  and  September  1985.  The 
number  of  En  Route  Screen  program  entrants  more 
than  doubled  to  8,536,  while  the  number  of  Terminal 
Screen  program  entrants  rose  approximately  20%  to 
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4,997.  The  structure  of  the  screening  programs  was 
not  changed  to  accommodate  the  strike  recovery 
(Boone,  1984). 

Following  the  strike,  the  pass  rates  in  both  the  En 
Route  and  Terminal  Screen  programs  declined  (Man¬ 
ning,  Kegg,  &  Collins,  1989).  Between  August  1981 
and  September  1985,  entrants  to  the  En  Route  Screen 
program  had  an  average  pass  rate  of  52%,  compared 
with  the  pre-strike  pass  rate  of  67%.  En  Route  Screen 
failures  represented  36%  and  withdrawals  represented 
1 2%  of  the  total  8,536  entrants.  In  the  Terminal  Screen 
program,  the  post-strike  pass  rate  was  68%,  compared 
with  the  pre-strike  rate  of  74%.  Terminal  Screen  pro¬ 
gram  entrant  failures  represented  26%.  The  Terminal 
Screen  program  withdrawal  rate  was  6%  of  total  entrants. 

Examination  of  the  demographic  data  from  pre-  and 
post-strike  FAA  Academy  entrants  (Collins,  Manning 
&  Taylor,  1984)  revealed  a  shift  in  the  proportion  of 
entrants  reporting  prior  ATC  experience.  Prior  to  the 
strike,  70%  of  FAA  Academy  entrants  reported  either 
aviation  or  air  traffic  control  experience  prior  to  entry. 
After  the  strike,  nearly  two-thirds  of  the  entrants  had  no 
prior  aviation-related  experience.  FAA  Academy  pass 
rates  were  higher  for  the  group  of  entrants  with  prior 
ATC  experience  than  entrants  with  no  prior  experience 
(Collins,  Manning,  &  Taylor,  1 984) .  The  decline  in  the 
percentage  of  entrants  with  previous  ATC  experience, 
particularly  prior  military  controllers,  may  have  affected 
the  pass  rates.  Because  the  initial  qualification  courses 
were  designed  for  individuals  with  no  prior  experience, 
this  shift  in  demographics  was  manageable.  Table  3-1 
presents  demographics  for  each  of  the  time  periods 
examined  in  this  paper. 

The  “Non-radar  Screen,”  1986-1992 

Changes  to  the  program  during  the  first  decade 
involved  such  modifications  as  alterations  in  scoring 
and  the  number  of  graded  problems  (Manning,  Kegg,  & 
Collins,  1989).  In  1985,  however,  a  major  change  was 
made  to  consolidate  the  En  Route  and  Terminal  Screen 
programs  into  a  single  FAA  Academy  program,  the 
“Non-radar  Screen.”  Analyses  of  the  longitudinal  data¬ 
bases  had  begun  to  point  to  areas  in  which  the  screening 
effectiveness  and  cost  benefits  could  be  enhanced.  Be¬ 
cause  the  En  Route  Screen  program  was  found  to  better 
correlate  with  measures  of  field  success  such  as 
supervisor’s  ratings  (V anDeventer,  1981),  than  the  Ter¬ 
minal  program,  the  En  Route  Screen  program  was 
selected  as  the  basis  for  the  Non-radar  Screen  (Manning, 
Kegg,  &  Collins,  1989).  Some  changes  were  made  to  the 


weights  assigned  to  the  scores  on  various  components  to 
achieve  a  projected  60%  pass  rate.  The  change  to  using 
a  single  screening  course  meant  that  students  were  not 
assigned  to  an  option  or  facility  until  they  completed  the 
Non-radar  Screen  and  their  FAA  Academy  scores  could 
be  used  to  place  graduates.  It  was  anticipated  that 
assigning  students  with  higher  aptitude  to  more  com¬ 
plex  facilities  would  result  in  further  reductions  in  field 
training  attrition.  In  addition,  the  Non-radar  Screen 
was  re-sequenced  in  Fiscal  Year  1989  to  be  the  first 
course  completed  by  FAA  Academy  students.  It  was 
anticipated  that  by  postponing  the  three-week,  aca¬ 
demic  phase  of  training  that  originally  preceded  the 
pass/fail  Non-radar  Screen,  the  training  costs  associated 
with  people  who  would  fail  or  withdraw  from  the  screen 
would  be  avoided  through  the  re-sequencing.  The  nine- 
week  Non-radar  Screen  was  implemented  in  October 
1985  (FY  1986)  and  continued  through  the  January 
1992  entering  class.  Non- radar  Screen  pass  rates  were 
56.6%  for  14,392  entrants.  Failures  represented  32.2%, 
and  11.2%  withdrew. 

Validity  Evidence 

Over  the  years,  several  studies  examined  various 
measures  to  assess  the  predictive  validity  of  the  FAA 
Academy  screening  programs.  These  included  attrition 
rates,  supervisor  ratings,  and  field  training  progress. 
VanDeventer  (1981)  reported  a  correlation  of  .56  (ad¬ 
justed  for  explicit  restriction  in  range  on  FAA  Academy 
score)  between  the  FAA  Academy  final  composite  score 
and  the  field  supervisor’s  rating  for  the  en  route  option. 
Manning,  Della  Rocco,  and  Bryant  (1989)  used  field¬ 
training  status  as  well  as  field  instructor  rating  as  crite¬ 
rion  measures.  They  reported  adjusted  correlations 
between  the  FAA  Academy  composite  score  and  in¬ 
structor  ratings  of  .46  and  .30  for  the  En  Route  and 
Terminal  Screen  programs,  respectively.  In  the  first 
report  of  data  on  Non-radar  Screen  graduates,  Della 
Rocco,  Manning,  and  Wing  (1990)  reported  an  ad¬ 
justed  correlation  of  .44  between  FAA  Academy  com¬ 
posite  score  and  field  training  status  of  graduates  assigned 
to  the  En  Route  option.  In  1994,  Broach  and  Manning 
reported  evidence  that  the  Non- radar  Screen  was  a  valid 
predictor  of  FAA  Academy  radar  training  performance. 

To  evaluate  the  final  status  of  the  Non-radar  Screen 
program,  correlations  between  the  FAA  Academy  com¬ 
posite  score  and  field  training  status  from  the  CAMI 
tracking  database  for  FAA  Academy  graduates  who 
entered  as  competitive  hires  were  examined.  Field  status 
was  a  variable  that  identified  ATCSs  in  their  first  field 
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facility  as  follows  (Manning,  1991):  1  =  Reached  Full 
Performance  Levely  2  =  Still  in  training  (Developmental) y 
3  =  Switched  facilities,  4  =  Switched  options,  or  5  =  Failed. 
Data  for  individuals  who  separated  from  the  occupation 
for  reasons  unrelated  to  performance  were  excluded. 
The  correlation  between  Non-radar  Screen  scores  and 
field  status  in  the  en  route  option  was  -.12  (A^=  3,484, 
p  <  .001).  Adjusted  for  explicit  restriction  in  range  on 
the  FAA  Academy  score  (Thorndike,  1949),  this  corre¬ 
lation  was  -.25.  The  correlation  between  Non-radar 
Screen  scores  and  terminal  field  status  was  -.08  {N  = 
2,505, <  .001).  The  correlation,  adjusted  for  range 
restriction,  was  -.17.  Figures  3-1  and  3-2  present  the 
relationship  between  FAA  Academy  Score  and  attain¬ 
ment  of  Full  Performance  Level  (FPL)  field  status  at  the 
first  facility  assignment  for  en  route  and  terminal  op¬ 
tions,  respectively. 

Evaluation  of  Potential  Adverse  Impact 

The  screening  programs  were  monitored  on  an  ongo¬ 
ing  basis  for  the  success  of  minorities  and  women  to 
determine  compliance  with  the  Uniform  Guidelines  on 
Employee  Selection  Procedures  (('Uniform  Guidelines)' 
Equal  Employment  Opportunity  Commission,  1978). 
The  Uniform  Guidelines  provided  a  “rule  of  thumb”  for 
assessing  adverse  impact  in  protected  groups  to  deter¬ 
mine  if  the  procedure  differentially  selected  one  group 
over  another.  An  impact  ratio  is  calculated  for  each 
target  group’s  pass  rate  by  the  pass  rate  of  the  group  with 
the  highest  rate  (typically  the  non-minorities).  By  the 
rule  of  thumb,  if  the  impact  ratio  is  greater  than  or  equal 
to  .80,  the  program  is  usually  assumed  to  demonstrate 
no  adverse  impact.  The  En  Route  and  Terminal  Screen 
programs  were  found  to  have  adverse  impact  against 
African  Americans  in  1984-85  (Manning,  Kegg,  & 
Collins,  1989).  Part  of  the  rationale  for  modifying  the 
FAA  Academy  program  scoring  was  to  reduce  the  ad¬ 
verse  impact.  Initial  assessments  of  the  Non-radar  Screen 
program  for  the  January  1986-January  1988  entrants 
revealed  an  impact  ratio  of  .832,  an  improvement  over 
the  old  En  Route  Screen  program.  However,  for  the 
final  two  and  a  half  years  of  the  program  the  ratio  for 
African  Americans  fell  below  the  .80  rule  of  thumb.  A 
review  of  impact  ratios  by  fiscal  year  revealed  the  follow¬ 
ing:  .55  for  FY  90,  .60  for  FY  91,  and  .52  for  the  final 
classes  during  FY  92. 


Discussion 

The  two-stage  ATCS  selection  process  involving  the 
Office  of  Personnel  Management  (formerly  Civil  Ser¬ 
vice  Commission)  testing  and  FAA  Academy  screening, 
was  a  very  robust  and  predictable  system  that  served  the 
agency  well  as  a  gateway  to  a  safety-critical  career.  The 
experience  with  the  two-stage  system,  as  well  as  a  data¬ 
base  for  assessing  system  performance,  provided  a  foun¬ 
dation  for  recovery  from  the  PATCO  strike  in  1981. 

The  correlations  in  the  present  analyses  between 
Non-radar  Screen  scores  and  field  status  were  lower  than 
those  previously  reported.  One  possible  reason  was  that 
field  training  was  modified  during  the  late  1 980s  (Man¬ 
ning,  1991).  The  influx  of  a  large  number  of 
developmental  at  the  lower-staffed  facilities  would 
certainly  have  led  to  some  adjustments  in  the  processes 
needed  to  provide  instruction  and  supervision  for  the 
on-the-job  training  provided  to  the  developmentals.  To 
date,  adequate  criterion  measures  have  been  a  problem 
in  establishing  the  validity  of  the  ATCS  screening 
programs  (Della  Rocco,  Manning,  &  Wing,  1990; 
Broach  &  Manning,  1 997) .  The  decline  in  impact  ratios 
may  have  resulted  from  the  direct  hire  authority  granted 
to  the  FAA  by  OPM  during  the  late  1 980s  (Della  Rocco 
&  Sawyer,  1990).  A  higher  percentage  of  applicants 
with  lower  OPM  scores  were  hired  with  a  predictable 
outcome  of  lower  FAA  Academy  pass  rates. 

In  March  1992,  the  final  classes  graduated  from  the 
FAA  Academy  Non-radar  Screen  program,  marking  the 
end  of  an  era  for  air  traffic  control  specialist  selection 
procedures,  which  had  lasted  nearly  two  decades.  The 
system  succumbed  to  various  pressures,  such  as  calls  to 
shorten  and  reduce  the  costs  of  the  second  stage  screen 
(Aerospace  Sciences,  Inc.,  1991),  as  well  as  a  compro¬ 
mise  of  the  OPM  test  battery  (Manning,  1991).  In  the 
Screen’s  miniaturized  training-testing  program,  indi¬ 
viduals  with  no  prior  knowledge  of  the  ATC  occupation 
demonstrated  that  they  could  separate  airplanes  in  labo¬ 
ratory  simulations  prior  to  continuing  in  the  career 
field.  The  FAA  is  currently  examining  the  possibility  of 
using  a  computerized  test  battery  to  replace  the  former 
selection  systems.  It  remains  to  be  seen  whether  or  not 
a  cognitive  test  battery,  which  measures  component 
skills  and  abilities  in  combination  with  noncognitive 
measures,  can,  in  fact,  effectively  replace  the  work 
sample  model  employed  in  the  screening  programs.  The 
history  of  ATCS  selection  certainly  suggests  that  a 
robust  system  can  be  developed  to  supply  a  steady  flow 
of  qualified  applicants. 
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Figure  3-1:  Percent  achieving  en  route  FPL  field  status  by  FAA  Academy  score 


Figure  3-2:  Percent  achieving  terminal  FPL  field  status  by  FAA  Academy  score 
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Table  3-1 .  Demographic  profile  of  FAA  Academy  entrants 


Pre-Strike 

1976-1981 

(A/=6,059) 

Post-Strike 

1981-1985 

(/V=1 3,533) 

Screen  1986-1992 

(N=1 4,392) 

Experience 

No  related  experience 

30% 

65% 

79% 

Aviation  Only 

20% 

17% 

5% 

ATC  Operations 

50% 

18% 

16% 

Average  Age 

26.5 

26.5 

26.2 

Education 

High  School  Only 

14% 

9% 

11% 

Some  College 

48% 

44% 

54% 

College  Degree 

38% 

47% 

35% 

Gender 

Male 

83% 

85% 

80% 

Female 

17% 

15% 

20% 

Minority  Status 

Minority 

13% 

8% 

12% 

Non-Minority 

87% 

92% 

88% 

22 


CHAPTER  4: 

AIR  TRAFFIC  CONTROL  SPECIALIST  FIELD  TRAINING  PROGRAMS,  1981-1992 

Carol  A.  Manning 

FAA  Civil  Aeromedical  Institute,  Oklahoma  City,  Oklahoma 


This  paper  presents  statistics  describing  the  status  of  graduates  from  the  FAA  Academy  ATCS  screening  programs 
(Della  Roccoy  this  volume)  in  subsequent  field  training  programs.  Of  the  8, 1 60  FAA  Academy  graduates  assigned  to 
En  Route  facilities  between  1981  and  1992,  71.6%  were  certified  as  full performance  level  (FPL)  controllers  by  June 
1995.  Of  the  6,417  FAA  Academy  graduates  assigned  to  Terminal  facilities  between  1981  and  1992,  83.8%  were 
certified  as  FPL  by  June  1995.  The  data  indicate  that  the  FAA  successfully  implemented  and  executed  a  large-scale 
field  training  program  to  rebuild  the  controller  workforce  following  the  1981  PATCO  strike. 


When  approximately  10,400  of  17,000  air  traffic 
controllers  were  fired  as  a  result  of  the  1981  Professional 
Air  Traffic  Controller  Organization  (PATCO)  strike, 
the  FAA  was  required  to  hire  and  train  their  replace¬ 
ments.  Other  papers  in  this  volume  describe  the  pro¬ 
cesses  for  recruiting  and  selecting  Air  Traffic  Control 
Specialist  (ATCS)  trainees.  This  paper  presents  statistics 
describing  their  status  in  ATCS  field  training  programs. 
The  statistics  were  extracted  from  a  data  base  developed 
at  the  FAA  Civil  Aeromedical  Institute  (CAMI)  after 
the  1981  strike.  The  data  were  used  to  track  develop¬ 
mental  progress  in  field  training  (per  FAA  Order 
3 120.22A,  1 985)  and  provide  feedback  about  the  effec¬ 
tiveness  of  the  selection  procedures.  The  complete  field 
training  data  base  included  information  about  those 
who  graduated  from  the  FAA  Academy  and  certain 
other  types  of  trainees  (e.g.,  military  hires)  regarding 
overall  success  rates  in  field  training  as  well  as  pass  rates 
and  times  to  complete  specific  phases  of  field  training. 
Only  field  training  status  for  FAA  Academy  graduates 
will  be  described  here. 

Background 

Successftil  graduates  of  the  FAA  Academy  in  Okla¬ 
homa  City  entered  a  multi-phase  training  program 
administered  at  their  assigned  field  facility.  The  content 
of  this  training  depended  on  their  option  (En  Route  or 
Terminal)  and  facility  assignment.  For  several  years 
after  the  strike,  trainees  assigned  to  the  en  route  option 
(that  is,  Air  Route  Traffic  Control  Centers  or  ARTCCs) 
took  equivalent  types  of  training  that  differed  only 
according  to  their  area  of  specialization.  Each  ARTCC 


partitions  their  airspace  into  between  4  and  8  areas  of 
specialization.  A  developmental  who  began  training  at 
an  ARTCC  was  assigned  to  only  1  of  these  areas  and  was 
required  to  certify  that  he  or  she  could  perform  air  traffic 
control  (ATC)  duties  on  all  sectors  (or  sub-partitions  of 
airspace)  within  that  area  before  being  allowed  to  con¬ 
trol  traffic  without  supervision.  After  an  initial  phase  of 
training  in  which  trainees  demonstrated  their  knowl¬ 
edge  of  the  area  of  specialization  by  learning  the  maps 
for  all  sectors,  they  took  a  series  of  training  phases 
designed  to  prepare  them  for  certification  on  3  types  of 
ATCS  duties  (those  of  the  assistant  controller,  radar 
associate/non-radar  controller,  and  radar  controller)  in 
their  area.  Training  for  one  set  of  ATC  duties  was 
typically  completed  on  all  sectors  before  beginning 
training  for  the  next  set  of  duties.  Training  phases 
conducted  in  classroom,  laboratory,  and  OJT  (on-the- 
job  training)  environments  were  reported  separately. 

The  content  of  training  at  terminal  facilities  de- 
-pended  on  the  ATC  positions  used  at  each  type  of 
facility.  Non-approach  or  Visual  Flight  Rules  (VFR) 
facilities  conducted  training  on  Flight  Data,  Clearance 
Delivery,  Ground  Control,  and  Local  Control  posi¬ 
tions.  Limited  radar  facilities  conducted  an  additional 
phase  for  the  Non-radar  position.  Trainees  at  facilities 
that  had  both  non-approach  and  radar  functions  com¬ 
pleted  all  the  above  phases  along  with  an  additional 
phase  of  training  for  the  Radar  position.  Some  facilities 
were  split  administratively  so  that  controllers  worked  in 
either  the  tower  cab  or  the  radar  room  but  not  in  both. 
In  this  case,  they  were  required  to  take  only  phases  of 
training  relevant  to  their  specific  duties.  Each  phase  of 
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terminal  training  included  classroom,  laboratory,  and 
OJT  components.  Individuals  who  completed  their 
training  at  one  terminal  facility,  became  Full  Perfor¬ 
mance  Level  (FPL)  controllers,  then  transferred  to  an¬ 
other  facility,  were  required  to  complete  all  phases  of 
training  relevant  to  their  new  facility,  even  if  they  had 
already  completed  the  same  phases  at  their  previous 
facility.  This  process  was  required  to  accommodate 
learning  how  to  perform  ATC  duties  in  a  new  airspace 
at  a  new  facility  which  may  be  different  than  the  way 
those  duties  were  performed  in  the  airspace  at  the 
previous  facility. 

Sample  Demographics 

As  described  by  Della  Rocco  (this  volume),  the 
demographics  of  candidates  entering  the  FAA  Academy 
screening  programs  changed  between  1981  and  1992, 
so  it  can  be  expected  that  the  demographics  of  those 
entering  field  training  would  reflect  those  changes. 
Table  4-1  compares  demographics  for  graduates  of  the 
1981-1985  En  Route  Screen  program  (hereafter  re¬ 
ferred  to  as  ‘the  En  Route  Screen’),  the  1981-1985 
Terminal  Screen  program  (hereafter  referred  to  as  ‘the 
Terminal  Screen’)  and  the  1 986- 1 992  Non-radar  Screen 
program  (hereafter  referred  to  as  ‘the  Non-radar  Screen’) . 

Differences  in  demographics  can  be  seen  between 
groups  of  graduates  of  the  two  FAA  Academy  programs 
conducted  at  the  same  time  as  well  as  groups  that 
graduated  at  different  times.  The  percentage  of  males 
graduating  from  the  En  Route  Screen  was  slightly 
higher  (88%)  than  was  the  percentage  graduating  from 
either  the  Terminal  Screen  (83%)  or  from  the  Non¬ 
radar  Screen  program  (82%).  The  percentage  of  non¬ 
minorities  was  highest  among  En  Route  Screen  graduates 
(95%),  lower  among  Terminal  Screen  graduates  (about 
92%),  and  lowest  among  graduates  of  the  Non-radar 
Screen  (about  91%).  Most  trainees  graduating  from  the 
FAA  Academy  entered  through  the  competitive  hiring 
process.  Over  90%  of  En  Route  Screen  graduates  were 
competitive  hires,  while  about  the  same  percentage  of 
Terminal  and  Non-radar  Screen  graduates  (82.4%  and 
82.7%,  respectively)  were  hired  competitively.  Of  those 
for  whom  the  amount  of  education  was  known,  about 
the  same  percentage  of  En  Route  and  Terminal  Screen 
graduates  had  a  college  degree  (46.7%  and  48.4%, 
respectively)  while  fewer  Non-radar  Screen  program 
graduates  had  a  college  degree  (about  35%). 


Field  Training  Outcomes 

Developmental  controllers  who  entered  field  train¬ 
ing  after  the  1981  strike  were  required  to  complete  each 
phase  of  training  successfully  in  a  specified  period  of 
time  to  avoid  removal  (FAA  Order  3330.30C,  1984).  If 
removed  from  training,  their  disposition  depended  on 
the  type  of  facility  to  which  they  were  assigned  and  the 
amount  of  training  they  completed  prior  to  removal. 
Generally,  en  route  trainees  who  completed  part  of  their 
radar  training  prior  to  removal  were  eligible  for  reassign¬ 
ment  to  a  terminal  facility  of  level  3  or  below.  En  route 
trainees  completing  initial  radar  associate  training  prior 
to  removal  were  eligible  for  reassignment  to  a  non-radar 
facility  or  a  Flight  Service  Station  (FSS).  Those  who 
completed  the  classroom  portion  of  radar  associate 
training  were  eligible  for  reassignment  to  an  FSS.  A 
somewhat  similar  reassignment  structure  existed  for 
terminal  trainees.  Those  assigned  to  a  level  4  or  5  facility 
who  failed  to  complete  training  on  local  control  were 
eligible  for  reassignment  to  a  level  1  or  2  non-radar  or 
limited  radar  facility.  Those  who  failed  to  complete 
radar  training  were  eligible  for  reassignment  to  a  lower 
level  non-radar  facility  or  level  2  limited  radar  facility. 
Those  who  failed  to  complete  radar  training  after  certi¬ 
fying  on  at  least  1  radar  position  were  eligible  for 
reassignment  to  a  lower  level  terminal  facility. 

A  variable  was  developed  to  describe  the  status  in 
field  training  for  trainees  at  their  first  facility  assignment 
after  FAA  Academy  graduation.  Use  of  this  variable 
assumed  that  a  developmental  who  transferred  to  an¬ 
other  facility  before  reaching  FPL  status  did  so  because 
of  failure  to  complete  the  training  program.  Exceptions 
to  this  assumption  were  made  for  those  that  we  knew 
had  transferred  or  separated  from  the  occupation  for 
reasons  unrelated  to  failure  and  for  en  route  trainees 
who  switched  to  another  center  before  reaching  FPL 
status  because  this  type  of  transfer  was  typically  unre¬ 
lated  to  training  performance.  However  other  transfers, 
for  example,  from  en  route  to  a  lower  level  terminal 
facility  or  to  a  FSS,  or  from  a  terminal  radar  facility  to 
a  lower  level  terminal  facility  or  a  FSS,  were  assumed  to 
be  related  to  training  failure. 

The  values  of  the  variable  indicating  field  training 
status  are  shown  in  Table  4-2.  Facility  switches  within 
the  terminal  option  were  typically  from  a  higher  activity 
facility  to  a  lower  activity  facility  and  typically  resulted 
from  failure  to  complete  training.  On  the  other  hand, 
facility  switches  within  the  en  route  option  were  not 
considered  failures  because  en  route  failures  typically 
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resulted  in  reassignments  across  options  and  not  to 
facilities  within  the  same  option.  Thus,  it  was  consid¬ 
ered  appropriate  to  categorize  field  training  status  sepa¬ 
rately  for  each  option. 

Changes  in  Field  Training  Status  Over  Time 

Tables  4-3  and  4-4  show  status  in  en  route  and 
terminal  field  training  by  the  FAA  Academy  program 
attended.  About  10%  more  graduates  of  the  Non-radar 
Screen  program  reached  FPL  status  than  did  graduates 
of  either  the  En  Route  or  Terminal  Screens.  Table  4-3 
also  shows  that  a  slightly  higher  percentage  of  Non¬ 
radar  Screen  graduates  (about  9%)  than  En  Route 
Screen  program  graduates  (about  7%)  transferred  to 
terminal  facilities.  Flowever,  the  percentage  of  transfers 
to  FSS  as  well  as  failures  was  lower  for  Non- radar  Screen 
graduates  than  for  graduates  of  the  En  Route  Screen. 
Also,  a  small  percentage  of  Non-radar  Screen  program 
graduates  had  not  completed  their  training  when  these 
analyses  were  conducted  in  April  of  1997. 

Graduates  of  the  Non-radar  Screen  program  who 
entered  terminal  facilities  had  a  lower  percentage  of 
facility  switches,  FSS  transfers,  and  failures  than  did 
graduates  of  the  Terminal  program.  Additional  analyses 
of  field  training  status  by  year  of  graduation  showed  a 
systematic  increase  in  rates  of  reaching  FPL  over  time. 
For  FAA  Academy  graduates  entering  en  route  field 
training,  rates  of  reaching  FPL  status  were  63-66%  for 
1982-84  classes,  70-75%  for  1985-88  classes,  and  80- 
83%  for  1 989-9 1  classes.  Only  about  72%  of 1 992  FAA 
Academy  Non-radar  Screen  program  graduates  in  en 
route  field  training  reached  FPL  status  as  of  April  1997, 
but  an  additional  1 3%  of  that  group  are  still  in  training. 
(Although  previous  studies  found  that  the  average  time 
to  reach  FPL  status  in  the  En  Route  option  is  about  3 
years  [e.g..  Manning,  Della  Rocco,  &  Bryant,  1989], 
some  individuals  were  identified  who  remained  in  train¬ 
ing  for  6  years  or  more.)  For  those  entering  terminal 
field  training,  rates  of  reaching  FPL  status  were  76-82% 
for  1982-84  classes,  85-89%  for  1985-89  classes,  and 
89-92%  for  1990-92  classes. 

The  causes  of  observed  changes  in  rates  of  attrition 
from  training  over  time  are  difficult  to  determine  be¬ 
cause  modifications  occurred  in  the  content  of  both  the 
FAA  Academy  Non-radar  Screen  and  in  field  training. 
The  Non-radar  Screen  program,  introduced  in  1986, 
contained  less  information  about  weather  and  aircraft 
performance  characteristics  than  did  the  former  Termi¬ 
nal  and  En  Route  Screens.  Subsequent  training  (both  at 
the  FAA  Academy  and  in  the  field)  was  eventually 


modified  to  incorporate  those  lessons.  It  might  be 
expected  that  removing  lessons  from  the  Non-radar 
Screen  program  would  result  in  an  increased  training 
attrition  rate.  However,  training  attrition  rates  actually 
decreased  for  Non-radar  Screen  graduates.  Thus,  changes 
in  the  content  of  the  Non-radar  Screen  program  were 
probably  not  responsible  for  the  changes  in  field  attri¬ 
tion  rates  for  Non-radar  Screen  program  graduates. 

Another  factor  that  may  have  reduced  attrition  from 
en  route  field  training  is  a  change  in  the  content  of  the 
field  training  program,  en  route  field  training  was 
changed  at  some  facilities  to  decrease  the  emphasis  on 
non-radar  air  traffic  control  duties  and  increase  the 
emphasis  on  radar  associate  duties.  Although  a  certain 
amount  of  non-radar  training  had  been  required  for  all 
trainees,  it  was  not  directly  applicable  to  the  AT C  duties 
that  some  controllers  actually  performed.  It  was  deter¬ 
mined  that  the  amount  of  non-radar  training  provided 
should  be  proportional  to  the  type  of  control  that  would 
be  conducted  by  the  developmental  controllers.  This 
change,  which  made  the  training  more  relevant,  may 
also  have  resulted  in  a  reduction  in  the  attrition  rate 
because  a  high  percentage  of  failures  could  be  traced 
back  to  non-radar  training. 

Some  en  route  facilities  also  changed  the  order  in 
which  training  phases  were  administered  so  that  train¬ 
ing  for  all  control  activities  (radar  associate  and  radar 
control)  was  completed  for  a  pair  of  sectors  before 
moving  on  to  any  other  sectors.  Additionally,  informal, 
undocumented  changes  occurred  in  the  administration 
of  field  training  as  a  result  of  a  decrease  in  the  number 
of  people  entering  the  facilities  after  about  mid- 1983. 
Prior  to  the  strike,  numbers  of  trainees  entering  the 
system  (701  in  1980  and  300  before  August  1981)  were 
much  lower  than  the  first  2  years  after  the  strike  (3416 
in  1982  and  1720  in  1983).  Lower  rates  of  failure  and 
transfers  to  other  facilities  might  have  occurred  for 
trainees  who  graduated  from  the  FAA  Academy  after 
1983  because  their  OJT  instructors  had  more  time  to 
spend  with  individual  trainees. 

Field  Training  Status  and  Demographics 

Tables  4-5  and  4-6  show  field  training  status  for  each 
option  by  the  demographic  variables  described  above. 
Table  4-5  shows  field  training  status  by  demographic 
groups  for  those  entering  en  route  facilities.  Rates  of 
reaching  FPL  status  were  approximately  equivalent  for 
male  (71.4%)  and  female  (72.5%)  en  route  trainees. 
African-American  trainees  had  lower  rates  of  reaching 
FPL  (about  63%)  than  did  non-minorities  (about  72%) , 
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but  were  slightly  more  likely  to  be  transferred  to  an  FSS 
(about  11%)  than  were  non-minority  trainees  (6%). 
Other  minority  groups  had  rates  of  reaching  FPL  status 
that  were  the  same  or  better  than  were  those  for  non¬ 
minorities.  FAA  Academy  graduates  who  were  non¬ 
competitive  and  Pre-Developmental  entrants  were  less 
likely  to  reach  FPL  status  (about  62%  and  55%,  respec¬ 
tively)  than  were  competitive  entrants  (about  73%). 
FAA  Academy  graduates  from  the  Airway  Science  pro¬ 
gram  had  a  higher  rate  of  reaching  FPL  (95%)  than  did 
competitive  entrants  but  their  overall  numbers  were 
low.  Those  with  any  type  of  college  degree  were  some¬ 
what  less  likely  to  become  FPL  (81%)  than  were  those 
without  a  college  degree  (86%). 

Table  4-6  shows  similar  statistics  for  terminal  train¬ 
ees.  Females  were  somewhat  less  likely  to  reach  FPL 
status  (80%)  in  a  terminal  facility  than  were  males 
(about  85%)  and  were  more  likely  than  males  to  switch 
facilities  within  the  terminal  option  (1 1%  as  compared 
with  7%).  Hispanic  (80%),  and  to  a  greater  extent, 
African-American  trainees  (77%)  were  less  likely  than 
were  non-minority  trainees  (84%)  to  reach  FPL  status. 
Of  those  who  failed  to  reach  FPL  status,  Hispanics  were 
more  likely  to  be  reassigned  to  another  terminal  facility 
(about  13%)  than  were  non-minorities  (about  8%)  and 
African  Americans  (13%)  were  more  likely  to  fail  and 
separate  from  the  occupation  than  were  non-minorities 
(about  7%) .  FAA  Academy  graduates  who  went  through 
Airway  Science  programs  were  more  likely  to  reach  FPL 
(95%)  than  were  competitive  entrants  (85%).  Non¬ 
competitive  entrants  (75.5%)  and  Pre-Developmental 
entrants  (72%)  were  less  likely  than  were  competitive 
entrants  to  reach  FPL  status.  Again,  those  with  a  college 
degree  were  less  likely  to  reach  FPL  status  (8 1  %)  than 
were  those  without  a  college  degree  (86%). 

Time  Until  Occurrence  of  Training  Attrition 

Table  4-7  shows  the  average  time  for  attrition  from 
training  to  occur  by  option  and  type  of  FAA  Academy 
program.  Training  attrition  at  the  first  facility  occurred 
between  about  1  year  and  more  than  2  Vi  years  after  FAA 
Academy  graduation.  It  appears  that  en  route  trainees 
who  failed  to  complete  training  were  retained  at  their 
facility  longer  than  were  terminal  trainees.  However,  the 
categories  included  in  the  Type  of  Attrition  variable 
were  not  the  same  for  the  two  options  so  a  direct 
comparison  is  not  appropriate.  The  amount  of  time 
until  attrition  from  training  in  the  terminal  option  was 


the  same  or  shorter  for  Non-radar  Screen  graduates  than 
for  graduates  of  the  Terminal  Screen  for  all  types  of 
attrition  except  failure.  On  the  other  hand,  the  amount 
of  time  to  attrition  from  training  in  the  en  route  option 
was  consistently  longer  for  Non-radar  Screen  graduates 
than  for  graduates  of  the  En  Route  Screen  program, 
regardless  of  type  of  attrition. 

Conclusions 

The  administration  of  ATCS  field  training  programs 
was  affected  by  changes  that  occurred  during  the  years 
after  the  1981  controller  strike.  In  the  early  1980s,  there 
was  an  influx  of  new  trainees  that  tested  the  capabilities 
of  the  training  system.  Not  only  did  a  large  number  of 
trainees  need  to  use  the  limited  lab  space  at  the  same 
time,  there  were  also  few  instructors  available  to  provide 
OJT.  The  numbers  of  FAA  Academy  graduates  entering 
field  training  reduced  after  1983,  but  it  took  several 
years  for  the  large  group  of  1981-83  trainees  to  make  it 
through  the  bottlenecks  in  the  training  system  to  com¬ 
plete  training.  It  is  likely  that  FAA  Academy  graduates 
who  entered  training  later  received  more  attention  than 
did  those  who  entered  training  immediately  after  the 
strike.  Data  regarding  the  length  of  time  until  attrition 
occurred  and  rates  of  reaching  FPL  status  support  this 
interpretation. 

A  more  efficient  second-stage  selection  procedure 
was  introduced  at  the  FAA  Academy  in  the  fall  of  1 985. 
At  the  same  time,  programs  to  recruit  under-represented 
groups  resulted  in  increases  in  the  percentages  ofwomen 
and  minorities  entering  field  training.  Women  suc¬ 
ceeded  in  training  in  about  the  same  proportion  as  men, 
while  minorities  had  somewhat  lower  rates  of  reaching 
FPL  than  did  non-minorities.  The  overall  result  should 
be  an  increase  in  the  diversity  of  the  ATC  workforce. 
Changes  were  made  in  the  content  of  en  route  field 
training  programs  after  about  1985.  These  changes 
resulted  in  a  training  program  that  better  targeted 
specific  skills  performed  on  the  job. 

The  percentage  of  trainees  reaching  FPL  status  in¬ 
creased  for  those  entering  field  training  during  the  late 
1980s.  Because  so  many  changes  occurred  in  both 
selection  procedures  and  training  programs  at  about  the 
same  time,  we  cannot  determine  why  the  success  rates 
increased.  However,  the  data  suggest  that  the  FAA 
successfully  accomplished  the  requirement  to  train  a 
substantial  number  of  new  air  traffic  controllers  to 
replace  the  fired  controllers. 
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Table  4-1:  Demographics  of  FAA  Academy  program  graduates 


FAA  Academy  Program 

En  Route  Screen 
1981-85 

Termina 

1981 

1  Screen 
-85 

Non-radar  Screen 
1986-92 

Group  Total 

N 

% 

N 

% 

N 

% 

N 

% 

Male 

3,928 

88.0 

2,814 

Gender 

83.0  6,683 

82.0 

13,425 

83.9 

Female 

535 

12.0 

576 

17.0 

1,465 

18.0 

2,576 

16.1 

Total 

4,463 

100.0 

3,390 

100.0 

8,148 

100.0 

1  6,001 

100.0 

Amer.  Indian 

19 

.4 

17 

Minority  Status*" 

.5  47 

.6 

83 

.5 

Asian-Pac. 

28 

.6 

32 

1.0 

98 

1.2 

158 

1.0 

African-Amer. 

81 

1.8 

119 

3.5 

339 

4.3 

539 

3.4 

Hispanic 

86 

2.0 

73 

2.2 

254 

3.2 

413 

2.6 

Non-Minority 

4,194 

95.1 

3,117 

92.8 

7,182 

90.7 

14,493 

92.4 

Total 

4,408 

100.0 

3,358 

100.0 

7,920 

100.0 

15,686 

100,0 

Competitive 

4,046 

92.7 

2,772 

Entry 

82.4 

Type' 

6,635 

82.7 

13,453 

85,4 

Airway  Science 

8 

.2 

4 

.1 

75 

.9 

87 

.6 

Pre-Devel. 

57 

1.3 

113 

3.4 

155 

1.9 

325 

2.1 

Coop.  Ed. 

34 

.8 

49 

1.5 

422 

5.3 

505 

3.2 

ATA 

43 

1.0 

94 

2.8 

564 

7.0 

701 

4.5 

VRA 

34 

.8 

41 

1.2 

21 

.3 

96 

.6 

Noncompetitive 

134 

3.1 

237 

7.0 

86 

1.1 

457 

2.9 

Former  ATC 

10 

.2 

54 

1.6 

61 

.8 

125 

.8 

Former  GS-9 

Total 

4,366 

100.0 

3,364 

100.0 

2 

8,021 

.0 

100.0 

2 

15,751 

.0 

100.0 

No  coll,  degree 

2,346 

53.3 

1,648 

Education^* 

51.6  5,230 

64.9 

9,224 

59.0 

College  degree 

2,053 

46.7 

1,544 

48.4 

2,824 

35.1 

6,421 

41,0 

Total 

4,399 

100.0 

3,192 

100.0 

8,054 

100.0' 

15,645 

100.0 

Notes:  "All  percents  are  for  columns  within  grouping. 

^Minority  status:  Amer.  Indian  =  American  Indian/Alaskan.;  Asian-Pac.  =  Asian-Pacific  Islander. 
"Entry  Type:  Airway  Science  =  graduate  of  accredited  Airway  Science  program.  Pre-Devel.  = 
participated  in  FAA  Pre-Developmental  Program;  Coop.  Ed.  =  Cooperative  Education  Program;  ATA 
—  former  Air  Traffic  Assistant  (GS-21 54  or  GS-21 02);  VRA  =  hired  under  Veterans  Readjustment 
Act;  Former  ATC  =  previously  graduated  from  FAA  Academy  and  failed  field  training;  Former  GS-9 
=  previously  hired  under  GS-9  military  hire  program  and  failed  field  training. 

‘^Education:  No  coll,  degree  includes  high  school  graduates  and  candidates  with  some  college  hour< 
but  no  degree.  College  degree  includes  those  with  a  bachelor's  or  higher  degree. 
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Option 

Status 

En  Route 

Terminal 

1 

FPL 

FPL 

2 

Developmental 

Developmental 

3 

Terminal  transfer 

Facility  switch 

4 

FSS  transfer 

FSS  transfer 

5 

Fail 

Fail 

rVOte.  rrL,  redcntJU  run  rci lui Lcvci.  w-v...  ...  . . Q 

at  first  facility.  Terminal  transfer,  transferred  to  Terminal  facility  before 
reaching  FPL  status.  Facility  switch,  transferred  to  another  Terminal  facility 
before  reaching  FPL  status.  FSS  transfer,  transferred  to  FSS  facility  before 
reaching  FPL  status.  Fail,  failed  training  and  separated  from  ATC 
occupation. 


Table  4-3:  En  Route  field  training  status  by  FAA  Academy  program 


FAA  Academy  Program’ 

En  route  Screen 
1981-85 

Non-radar  Screen 
1986-92 

Group  Total 

Field  Training  Status'” 

N 

% 

N 

% 

N 

% 

FPL 

2,758 

65.7 

3,405 

77.2 

6,163 

71.6 

Developmental 

48 

1.1 

48 

.6 

Terminal  transfer 

289 

6.9 

393 

8.9 

682 

7.9 

FSS  transfer 

424 

10.1 

115 

2.6 

539 

6.3 

Fail 

728 

17.3 

450 

10.2 

1,178 

13.7 

Group  Total 

4,199 

100.0 

4,411 

100.0 

8,610 

100.0 

Note:  ’All  percents  are  column  percents. 

‘’En  Route  field  training  status:  FPL,  reached  Full  Performance  Level.  Devel.,  Developmental.  Term, 
transfer,  transferred  to  Terminal  facility  before  reaching  FPL  status.  FSS  transfer,  transferred  to  FSS 
facility  Isefore  reaching  FPL  status.  Fail,  failed  training  and  separated  from  ATC  occupation. 
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Table  4-4;  Terminal  field  training  status  by  FAA  Academy  program 


FAA  Academy  Program’ 

Terminal  Screen 
1981-85 

Non-radar  Screen 
1986-92 

Group  Total 

Field  Training  Status" 

N 

% 

N 

% 

N 

% 

FPL 

Developmental 

2,538 

79.0 

2,840 

88.6 

5,378 

83.8 

Facility  switch 

305 

9.5 

215 

6.7 

520 

8.1 

FSS  transfer 

68 

2.1 

14 

0.4 

82 

1.3 

Fail 

301 

9.4 

136 

4.2 

437 

6.8 

Group  Total 

3,212 

100.0 

3,205 

100.0 

6,417 

100.0 

Note:  ’All  percents  are  column  percents. 

'Terminal  field  training  status:  FPL,  reached  Full  Performance  Level.  Facility  switch,  transferred  to 
another  Terminal  facility  before  reaching  FPL  status.  FSS  transfer,  transferred  to  r=c  ‘iJlUy  beiore 
reaching  FPL  status.  Fail,  failed  training  and  sen3'"’^°d  Ircxv.  ATr  <^cc.j(jcition. 
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Table  4-5:  En  Route  field  training  status  by  demographic  variables 


En  Route  field  training  status’ 

FPL 

Dev 

Term  Transfer 

FSS  Transfer 

Fail 

N 

% 

N 

% 

N 

% 

N 

% 

N 

% 

Gender 

Male 

5,292 

71.4 

34 

.5 

575 

7.8 

476 

6.4 

1,031 

13.9 

Female 

871 

72.5 

14 

1.2 

107 

8.9 

63 

5.2 

147 

12.2 

Minority  Status*" 

Amer.  Indian 

28 

77.8 

2 

5.6 

3 

8.3 

3 

8.3 

Asian-Pac. 

60 

84.5 

3 

4.2 

1 

1.4 

7 

9.9 

African-Amer. 

162 

63.3 

7 

2.7 

24 

9.4 

29 

11.3 

34 

13.3 

Hispanic 

160 

71.4 

1 

.4 

17 

7.6 

8 

3.6 

38 

17.0 

Non-minority 

5,630 

71.6 

40 

.5 

622 

7.9 

492 

6.3 

1,078 

13.7 

Entry  Type“ 

Competitive 

5,472 

72.9 

38 

.5 

557 

7.4 

473 

6.3 

971 

12.9 

Airway  Science 

31 

79.5 

3 

7.7 

1 

2.6 

4 

10.3 

Pre-Devel. 

76 

54.7 

1 

.7 

22 

15.8 

11 

7.9 

29 

20.9 

Coop.  Ed. 

191 

70.0 

4 

1.5 

25 

9.2 

10 

3.7 

43 

15.8 

ATA 

167 

60.7 

4 

1.5 

46 

16.7 

9 

3.3 

49 

17.8 

VRA 

29 

63.0 

1 

2.2 

2 

4.3 

4 

8.7 

10 

21,7 

Noncompetitive 

96 

61.5 

10 

6.4 

12 

7.7 

38 

24.4 

Former  ATC 

17 

68.0 

1 

4.0 

3 

12.0 

4 

16.0 

Former  GS-9 

1 

100.0 

Education*^ 

No  coll,  degree 

3,752 

74.0 

31 

.6 

400 

7.9 

267 

5.3 

620 

12.2 

College  degree 

2,367 

68.8 

17 

,5 

274 

8,0 

238 

6,9 

542 

15.8 

Note:  “Percents  are  row  percents. 

‘’Minority  status:  Amer.  Indian,  American  Indian/Alaskan.  Asian-Pac.,  Asian-Pacific  Islander.  Entry: 
Airway  Science,  graduate  of  accredited  Airway  Science  program.  Pre-Devel.,  participated  in  FAA 
Prg-Developmental  Program.  Coop.  Ed.,  Cooperative  Education  Program.  AT  A,  former  Air  Traffic 
Assistant  (GS-2154  or  GS-2102.)  VRA,  hired  under  Veterans  Readjustment  Act.  Former  ATC, 
previously  graduated  from  FAA  Academy  and  failed  field  training.  Former  GS-9,  previously  hired 
under  GS-9  military  hire  program  and  failed  field  training. 

“Education:  No  coll,  degree  includes  high  school  graduates  and  candidates  with  some  college  hours 
but  no  degree.  College  degree  includes  those  with  a  bachelor's  or  higher  degree. 


30 


Chapter  4:  Air  Traffic  Control  Specialist  Field  Training  Programs^  1981  -1992 


Table  4-6:  Terminal  field  training  status  by  demographic  variables 


Terminal  field  training  status" 

FPL 

Facility  switch 

FSS  transfer 

Fail 

N 

% 

N 

% 

N 

% 

N 

% 

Gender 

Male 

4434 

84.7 

390 

7.4 

66 

1.3 

347 

6.6 

Female 

944 

80.0 

130 

11.0 

16 

1.4 

90 

7.6 

Minority  Status'" 

Amer.  Indian 

33 

84.6 

2 

5.1 

4 

10.3 

Asian-Pac. 

63 

87.5 

3 

4.2 

2 

2.8 

4 

5.6 

African-Amer. 

197 

77.0 

22 

8.6 

4 

1.6 

33 

12.9 

Hispanic 

132 

80.0 

21 

12.7 

1 

.6 

11 

6.7 

Non-Minority 

4834 

84.1 

464 

8.1 

75 

1.3 

378 

6.6 

Entry  Type*^ 

Competitive 

4367 

84.8 

387 

7.5 

58 

1.1 

337 

6.5 

Airway  Science 

37 

94.9 

1 

2.6 

1 

2.6 

Pre-Devel. 

119 

72.1 

19 

11.5 

6 

3.6 

21 

12.7 

Coop  Ed 

164 

78.8 

24 

11.5 

1 

.5 

19 

9.1 

ATA 

309 

79.4 

48 

12.3 

4 

1.0 

28 

7.2 

VRA 

42 

89.4 

3 

6.4 

1 

2.1 

1 

2.1 

Noncompetitive 

182 

75.5 

28 

11.6 

5 

2.1 

26 

10.8 

Former  ATC 

83 

89.2 

4 

4.3 

4 

4.3 

2 

2.2 

Former  GS-9 

1 

100.0 

Education'* 

No  coll,  degree 

3157 

85.9 

282 

7.7 

25 

0.7 

210 

5.7 

College  degree 

2042 

81.3 

229 

9.1 

26 

1.0 

214 

8.5 

Note:  ^Percents  are  row  percents. 

“’Minority  status:  Amer.  Indian,  American  Indian/Alaskan.  Asian-Pac.,  Asian-Pacific  Islander.  'Entry: 
Airway  Science,  graduate  of  accredited  Airway  Science  program.  Pre-Devel.,  participated  in  FAA 
Pre-Developmental  Program.  Coop.  Ed.,  Cooperative  Education  Program.  AT A,  former  Air  Traffic 
Assistant  (GS-21 54  or  GS-2102.)  VRA,  hired  under  Veterans  Readjustment  Act.  Former  ATC, 
previously  graduated  from  FAA  Academy  and  failed  field  training.  Former  GS-9,  previously  hired 
under  GS-9  military  hire  program  and  failed  field  training. 

■“Education:  No  coll,  degree  includes  high  school  graduates  and  candidates  with  some  college 
hours  but  no  degree.  College  degree  includes  those  with  a  bachelor's  or  higher  degree. 
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Table  4-7:  Time  to  attrition  (years)  by  field  option,  FAA  Academy  program,  and  type  of  attrition 


Type  of  attrition 

FAA  Academy  Program 

En  Route  or  Terminal  Screen 
1981-85 

Non-radar  Screen 

1986-92 

N 

M 

SD 

N 

M 

SD 

Terminal 

Switch  FSS 

68 

1.65 

1.98 

14 

1.65 

0.62 

Fail 

300 

1.06 

1.05 

136 

1.22 

0.63 

Other 

169 

1.26 

1.42 

140 

0.94 

0.77 

All  types  attrition 

537 

1.20 

1.33 

291 

1.11 

0.72 

En  Route 

Switch  Terminal 

289 

2.20 

1.41 

392 

2.60 

1.33 

Switch  FSS 

423 

1.08 

0.98 

115 

1.84 

1.00 

Fail 

726 

1.17 

0.93 

449 

1.75 

1.02 

Other 

180 

1.18 

1.16 

281 

1.58 

1.26 

All  types  attrition 

1618 

1.33 

1.15 

1237 

1.99 

1.25 

Note:  Option:  Switch  FSS,  changed  options  from  either  En  Route  or  Terminal  to  Flight  Service 
Station.  Fail,  failed  training  and  separated  from  ATC  occupation.  Other,  separated  from 
ATC  occupation  for  reasons  unrelated  to  failure.  Switch  Terminal,  changed  options  from  En 
Route  to  Terminal. 
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CHAPTER  5: 

ORGANIZATIONAL  CLIMATE  PERCEPTIONS  OF  THE  AIR  TRAFFIC  CONTROL 
SPECIALIST  (ATCS)  WORKFORCE,  1981  1992 

Richard  C.  Thompson 

FAA  Civil  Aeromedical  Institute,  Oklahoma  City,  Oklahoma 

Five  organizational  climate  perceptions  of  Federal  Aviation  Administration  (FAA)  Air  Traffic  Control  Specialists 
(ATCS)  are  examined  from  1984  through  1990.  These  perceptions  of  job  satisfaction,  management  concern  for 
employees,  management  direction,  pay  and  benefits  equity,  and  organization  of work  were  found  in  previous  research 
to  he  related  to  the  1981  Professional  Air  Traffic  Controllers  Organization  (PATCO)  strike.  Because  of  this 
relationship,  FAA  developed  several  human  resources  initiatives  to  enhance  employee  perceptions  of  these  areas.  Trend 
data  for  the five  organizational  climate  dimensions  are  examined  during  the  course  of  rebuilding  the  ATCS  workforce. 
The  results  suggest  the  FAA  made  modest  improvements  between  1984  and  1988,  but  lost  some  ground  in  1990. 


On  August  3,  1981,  the  Professional  Air  Traffic 
Controllers  Organization  (PATCO)  engaged  in  a 
strike  against  the  Federal  Aviation  Administration 
(FAA).  The  strike  led  to  the  termination  of  a  large 
portion  of  the  Air  Traffic  Control  Specialist  (ATCS) 
workforce.  The  strike  also  led  to  an  examination  of 
the  organizational  climate  perceptions  of  the  FAA’s 
ATCS  workforce  Qones,  Bowers,  &  Fuller,  1984). 
Climate  perceptions  of  the  agency’s  employees  have 
been  obtained  biennially  since  that  initial  assessment 
as  part  of  the  agency’s  human  resource  initiatives 
developed  in  the  wake  of  the  strike.  The  purpose  of 
this  paper  is  to  describe  controller  perceptions  of 
organizational  climate  during  the  rebuilding  of  the 
ATCS  workforce. 

The  FAA  and  PATCO  had  a  difficult  relationship 
for  a  number  of  years.  The  strike  was  the  culmination 
of  this  conflict.  On  the  day  of  the  strike,  more  than 
10,000  controllers  walked  off  the  job.  President 
Reagan  gave  the  strikers  48  hours  to  return  to  work, 
which  served  as  a  cooling  off  period  and  a  deadline. 
The  FAA  implemented  a  strike  contingency  plan 
with  non-striking  employees,  military  controllers, 
and  supervisors  filling  in  for  the  strikers.  In  the  end, 
those  controllers  who  did  not  return  to  work  were 
terminated  and  the  union  was  decertified.  The  FAA 
had  to  quickly  replace  a  substantial  portion  of  its 
workforce,  find  out  why  the  strike  occurred,  and  if 
possible,  correct  the  problems. 


One  of  the  investigations  conducted  by  the  FAA 
focused  on  employee-management  conflict,  and  in¬ 
cluded  an  assessment  of  employee  perceptions.  The 
assessment  resulted  in  the  '']oncs  Report”  Qones, 
Bowers,  &  Fuller,  1982).  Jones  et  al.  identified  sev¬ 
eral  correlates  of  the  strike,  such  as  increasing  utiliza¬ 
tion  of  the  National  Airspace  System  (NAS),  episodic 
controller  stress,  equipment  failures,  poor  manage¬ 
ment/employee  relations  and  communication,  and, 
in  general,  low  morale.  The  Jones  Report  found  that 
morale  in  the  FAA  was  very  low,  even  at  the  upper 
levels  of  the  agency’s  hierarchy.  Morale  was  indicated 
by  the  summary  pattern  of  employee  perceptions 
about  the  agency.  Areas  of  particular  concern  in¬ 
cluded  satisfaction,  management  concern  for  em¬ 
ployees,  rewards  (pay  and  benefits),  organization  of 
work,  and  managerial  and  supervisory  direction. 

For  the  FAA  as  a  whole,  the  Jones  Report  results 
indicated  a  low  level  of  job  satisfaction.  Job  satisfac¬ 
tion  is  typically  treated  as  an  outcome  of  the  manage¬ 
ment  of  human  resources,  so  low  job  satisfaction 
scores  are  often  interpreted  as  problems  with  manage¬ 
ment  (Cranny,  Smith,  &  Stone,  1992).  Similarly,  the 
employees  perceived  that  management  did  not  con¬ 
sider  the  effects  of  decisions  on  employees,  suggest¬ 
ing  that  management  did  not  obtain  adequate 
employee  input.  Employees  also  perceived  the  job  to 
be  inadequate  in  terms  of  formal  and  informal  re¬ 
wards.  Instead,  the  perception  was  that  rewards  were 
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based  on  annual  performance  reviews  that  may  or 
may  not  have  been  related  to  actual  day-to-day  per¬ 
formance.  The  organization  of  work,  in  terms  of  the 
procedures  and  processes  involved  in  completing 
assigned  duties,  was  perceived  to  be  problematic  by 
controllers.  Finally,  managerial  and  supervisory  di¬ 
rection  was  rated  poorly  by  the  ATCS  workforce. 
Problems  with  supervisors  were  believed  to  be  prima¬ 
rily  due  to  the  autocratic  style  of  supervisors  selected 
from  the  ranks  of  controllers. 

The  perceptions  of  striking  and  non-striking  con¬ 
trollers  were  compared  in  the  Jones  Report.  The 
results  indicated  that  controllers  who  struck,  and 
were  subsequently  fired,  and  controllers  who  re¬ 
mained  had  a  similar  pattern  of  organizational  per¬ 
ceptions,  although  the  striking  controllers  generally 
indicated  lower  levels  of  most  perceptions.  In  addi¬ 
tion,  the  data  indicated  that  the  strike  occurred,  in 
part,  due  to  “...an  organization  climate  that  they 
[ATCSs]  experienced  as  uncaring,  unconcerned  for 
its  people,  uncommunicative,  and  unreceptive”  (Bow¬ 
ers,  1983,  p.  16).  In  response  to  these  conclusions  the 
FAA  developed  several  national  human  resource  ini¬ 
tiatives  that  targeted  the  morale  and  supervision 
problems  identified  in  the  Jones  Report.  These  initia¬ 
tives  included  survey-feedback  programs  to  respond 
to  employee  concerns,  improvement  of  the  selection 
and  human  resources  training  of  supervisors,  and 
changing  pay  systems. 

One  of  the  primary  FAA  responses  to  the  strike  was 
the  implementation  of  an  agency-wide  biennial  em¬ 
ployee  survey.  The  biennial  survey  was  created  to 
provide  management  with  a  picture  of  employee 
perceptions,  and  to  provide  a  means  of  tracking  those 
perceptions  over  time.  In  addition,  feedback  from  the 
survey  provided  management  with  information  re¬ 
garding  problem  areas  to  be  addressed.  Following  the 
first  administration  of  the  survey  in  1984,  interviews 
were  conducted  in  three  regions  to  further  clarify  why 
people  responded  they  way  they  did.  The  interviews 
indicated  the  areas  of  primary  concern  were  related  to 
human  resource  management  practices.  In  addition, 
regions  were  expected  to  formulate  action  plans  to 
address  human  resource  management  problems  within 
the  region.  The  regional  plans  were  then  combined 
into  a  national  plan  (Weithoner,  1985).  Human 
resources  specialists  were  hired  in  each  region  to 
further  support  efforts  to  initiate  plans  and  programs 
to  address  these  issues. 


After  the  1986  administration  of  the  agency- wide 
employee  survey,  the  feedback  and  action  planning 
process  that  had  emerged  from  an  earlier  administra¬ 
tion  was  augmented  and  formalized  in  the  Survey 
Feedback  Action  (SFA)  program.  The  SFA  relied  on 
two  surveys:  (1)  The  biennial  national  survey  de¬ 
scribed  above  that  focused  on  national,  agency  wide 
issues,  and  (2)  regional  surveys  focused  on  more 
specific  regional  issues.  The  regional  surveys  assessed 
employee  perceptions  of  supervisors  and  manage¬ 
ment.  Based  on  the  results  of  the  regional  surveys, 
regional  plans  were  developed  to  improve  training, 
communication,  and  supervision.  The  effectiveness 
or  impact  of  these  regional  action  plans  were  to  be 
evaluated  with  the  next  round  of  surveys. 

Another  problem  for  the  agency  that  resulted  from 
the  strike  was  under-staffing  of  ATCSs.  Understaffing 
was  an  immediate  problem  following  the  strike  be¬ 
cause  it  takes  as  long  as  3  years  after  hiring  to  train  a 
controller  to  Full  Performance  Level  (FPL; 
VanDeventer,  Collins,  Manning,  Taylor,  &:  Baxter, 
1984).  Understaffing  of  ATCSs  was  a  continuing 
problem  for  the  FAA  in  some  facilities  (General 
Accounting  Office,  1986,  1989).  To  enhance  re¬ 
cruitment  and  retention  of  employees,  the  DOT/ 
FAA  Pay  Demonstration  Project  (U.  S.  Office  of 
Personnel  Management,  1989)  was  implemented. 
This  project  provided  retention  bonuses  to  employ¬ 
ees  in  safety-related  occupations  such  as  ATCS,  elec¬ 
tronics  maintenance  technicians,  and  aviation  safety 
inspectors  at  difficult  to  staff  facilities. 

Another  change  that  occurred  due  to  the  strike  was 
the  influx  of  a  large  number  of  new  employees.  After 
the  termination  of  the  striking  controllers,  replace¬ 
ments  had  to  be  hired,  trained,  and  placed  on  the 
boards.  These  new  hires  differed  from  the  replaced 
controllers  in  terms  of  previous  experience  and  apti¬ 
tude  (VanDeventer,  Collins,  Manning,  Taylor,  & 
Baxter,  1984).  In  addition,  there  were  some  differ¬ 
ences  in  the  training  of  pre-and  post-strike  control¬ 
lers.  In  order  to  rapidly  replace  the  controllers,  the 
agency  hired  a  substantial  number  of  replacements 
“off  the  streets”  who  were  required  to  complete  the 
FAA  Academy  screen,  followed  by  continued  train¬ 
ing  at  the  FAA  Academy  and  in  the  field.  A  smaller 
number  of  former  military  controllers  were  hired  at 
higher  grade  levels  and  placed  directly  into  field 
facilities  for  on-the-job  training  (Manning,  Kegg,  & 
Collins,  1989;  Manning  &  Aul,  1992).  The  new 
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hires,  however,  were  placed  into  the  same  organiza¬ 
tional  climate  that  had  created  the  circumstances  that 
led  to  the  original  strike. 

A  final  program,  the  Supervisor  Identification  and 
Development  Program  (SIDP;  Myers,  1992),  was 
created  to  address  inadequate  supervisor/subordinate 
communication.  The  SIDP  was  created  initially  for 
the  air  traffic,  electronics  technician,  and  aviation 
safety  inspector  workforces.  The  purposes  of  SIDP 
were  to  enhance  the  quality  of  supervisors  selected  for 
internal  advancement  from  the  workforce,  and  to 
provide  improved  training  in  human  relations  super¬ 
visory  skills.  At  the  same  time,  the  FAA’s  Center  for 
Management  Development  changed  its  courses  to 
support  this  effort.  Again,  the  Jones  report  had  indi¬ 
cated  that  one  of  the  FAA's  problems  was  the  way  that 
management,  and  to  a  lesser  extent,  first  and  second 
line  supervisors,  treated  subordinates.  SIDP  con¬ 
sisted  of  a  self-selection  of  candidates,  who  were  then 
rated  by  peers  and  supervisors  on  four  performance 
dimensions:  interpersonal,  communications,  direc¬ 
tion  and  motivation,  and  technical  competence.  Can¬ 
didates  were  then  given  a  skill-based  interview.  From 
there,  candidates  were  considered  to  be  eligible  for 
promotion  consideration,  given  further  review,  or 
asked  to  enhance  their  leadership  skills  and  reapply 
(Myers,  1992). 

Given  the  continuing  efforts  of  the  FAA  to  im¬ 
prove  the  organizational  climate,  and  the  influx  of 
new  organization  members  following  the  strike,  it 
seems  prudent  to  assess  any  changes  in  the  percep¬ 
tions  of  agency  employees,  and  in  particular,  the 
perceptions  of  air  traffic  controller  specialists  (AT  CS) . 
The  four  national  initiatives,  the  biennial  survey, 
SFA,  the  Pay  Demonstration  Project,  and  SIDP, 
coupled  with  the  influx  of  new  employees,  may  have 
improved  the  organizational  climate  of  the  FAA. 
Indeed,  the  four  national  human  resource  initiatives 
were  developed  specifically  to  enhance  the  FAA's 
organizational  climate.  The  human  resource  initia¬ 
tives  undertaken  during  the  recovery  of  the  ATCS 
workforce  have  their  roots  in  the  conclusions  drawn 
by  the  Jones  Report.  The  extent  to  which  these 
projects  had  an  effect  on  employee  perceptions  will 
be  examined  next. 


Method 

Participants  and  procedures 

Biennial  surveys  of  varying  content  were  adminis¬ 
tered  to  FAA  employees  on  four  occasions  (1984, 
1986,  1988,  and  1990).  The  first  two  administra¬ 
tions  consisted  of  a  census  of  all  agency  employees. 
The  latter  two  administrations  consisted  of  a  strati¬ 
fied  random  sample  of  approximately  15%  of  the 
agency.  This  sample  consisting  of  air  traffic  control¬ 
lers  (Tower,  En  route,  or  Flight  Service  Station  per¬ 
sonnel)  and  supervisors  was  drawn  from  the  larger 
samples  of  the  FAA.  The  ATCS  samples  for  each 
administration  were:  12,478;  14,222;  2,106;  and 
2,485,  respectively. 

Measures 

Five  organizational  climate  dimensions  assessing 
content  areas  similar  to  those  identified  as  problem 
areas  by  Jones  et  al.  (1982)  were  examined.  Each 
dimension  was  assessed  with  the  same  items  for  each 
survey  administration,  except  for  job  satisfaction  and 
equity  of  pay  and  benefits.  The  initial  survey  in¬ 
cluded  a  single  item  measuring  job  satisfaction.  This 
single  item  job  satisfaction  measure  has  been  used  as 
a  benchmark  of  job  satisfaction  since  1 984.  The  three 
subsequent  surveys  included  additional  satisfaction 
measures  (satisfaction  with  pay,  benefits,  physical 
working  conditions,  work  group,  supervisor,  and 
organization)  that  were  averaged  to  form  a  satisfac¬ 
tion  index  (average  a  =  .77).  The  single  job  satisfac¬ 
tion  item  and  the  satisfaction  index  provide  two 
measures  of  overall  morale  in  the  agency.  Such  satis¬ 
faction  measures  are  commonly  used  as  outcome 
measures  in  studies  of  organization  change  (Cranny, 
Smith,  &  Stone,  1992).  The  second  measure  focuses 
on  perceptions  of  management’s  concern  for  employ¬ 
ees,  which  was  assessed  with  3  items  (average  a  =  .89) 
from  Jones  et  al.  (1982).  Management  consideration 
examines  the  extent  to  which  management  takes  into 
account  the  effects  of  decisions  on  employees.  Em¬ 
ployee  perceptions  of  pay  and  benefit  equity  (formal 
rewards)  were  initially  assessed  with  two  items  (e.g., 
pay  and  job  security  equity)  in  1984.  Two  additional 
items  (e.g.,  retirement  and  benefits  equity,  average  a 
=  .74)  were  incorporated  in  subsequent  surveys.  Per¬ 
ceptions  regarding  the  way  work  was  organized  were 
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assessed  by  4  items  (average  (X  =  .87).  The  amount  of 
direction  provided  by  management  was  assessed  by  3 
items  (average  a  =  .87).  Management  direction  in¬ 
cludes  things  such  as  setting  of  goals,  and  providing  the 
employees  with  appropriate  guidance.  For  all  of  the 
multiple-item  measures,  the  items  assessing  the  dimen¬ 
sion  were  averaged  to  obtain  a  single  dimension  score. 

Results 

A  one-way  AN OVA  demonstrated  there  was  a 
significant  trend  in  the  reporting  of  the  job  satisfac¬ 
tion  item,  F{3,  31164)  =  68.70, p  <  .001.  The  means 
and  standard  deviations  for  each  survey  administra¬ 
tion  are  presented  in  Table  5-1 .  As  can  be  seen,  there 
was  a  gradual  increase  in  job  satisfaction  between 
1984  and  1988,  and  a  slight  decrement  in  1990. 
However,  levels  of  job  satisfaction  in  1990  were  still 
above  those  noted  in  1984  and  1986.  Post  hoc 
analyses  indicate  all  means  are  significantly  different. 
Because  of  the  very  large  number  of  respondents, 
however,  any  difference  between  means  is  likely  to  be 
statistically  significant.  Therefore,  the  effect  size,  or 
the  amount  of  variance  accounted  for  was  examined 
(Cohen,  1988,  1992).  For  this  analysis,  the  amount 
of  variance  accounted  for  was  very  small  (T]^  <  .004). 
The  small  effect  size  is  due,  in  part,  to  the  single  item 
measure  of  job  satisfaction,  which  has  a  larger  stan¬ 
dard  deviation  compared  to  measures  based  on  mul¬ 
tiple  items,  as  can  be  seen  in  Table  5-1. 

To  determine  if  there  were  any  differences  in  the 
reported  perceptions  of  the  five  multiple  item  mea¬ 
sures  of  organization  climate  across  administrations 
a  one-way  MAN OVA  was  conducted.  This  test  indi¬ 
cated  the  presence  of  at  least  one  significant  differ¬ 
ence,  A(15,93024)=151.02,  p  <  .001.  Follow-up 
univariate  analyses  for  each  of  the  five  dependent 
variable  measures  were  also  statistically  significant. 
The  descriptive  statistics  for  this  analysis  are  also 
reported  in  Table  5-1 .  Again,  however,  because  of  the 
number  of  respondents,  effect  sizes  were  examined 
for  each  analysis  (Cohen,  1988,  1992).  With  a  single 
exception,  the  effect  size  of  each  analysis  was  trivial 
(e.g.,  <  .01).  The  exception  was  for  pay  and 

benefits  satisfaction  (Tj^  <  .042).  While  this  is  still  a 
small  effect,  an  examination  of  the  means  indicates 
that  the  level  of  perceived  equity  of  pay  and  benefits 
decreased  most  after  the  first  assessment. 


The  lack  of  discernible  upward  or  downward  trends 
in  the  above  analysis  led  to  an  examination  of  re¬ 
sponse  rates.  For  this  analysis,  the  individual  re¬ 
sponses  to  each  item  that  comprise  a  measure  were 
assessed  to  determine  what  proportion  of  responses 
can  be  categorized  into  disagree,  neither  agree  nor 
disagree,  or  agree  for  each  administration  of  the 
survey.  Figure  5-1  shows  the  trend  for  the  single  item 
measure  of  job  satisfaction.  As  can  be  seen,  there  is  a 
noticeable  upward  trend  from  1984  to  1988,  with  a 
drop  in  1990.  Figure  5-2  shows  there  is  a  bimodal 
distribution  of  responses  to  the  job  satisfaction  index 
with  just  over  50%  of  the  respondents  indicating 
satisfaction,  and  about  one  third  indicating  dissatis¬ 
faction.  In  1986,  1988,  and  1990  the  discrepancy 
between  the  job  satisfaction  item  and  the  satisfaction 
index  is  primarily  due  to  lower  ratings  of  benefits 
satisfaction,  and  to  a  lesser  extent,  to  items  dealing  with 
working  conditions  and  organization  satisfaction. 

The  percent  of  respondents  indicating  manage¬ 
ment  has  a  concern  for  employees  also  increases 
slightly  across  the  first  three  survey  administrations 
(see  Figure  5-3).  At  the  same  time,  the  number  of 
respondents  disagreeing  decreases.  A  different  pat¬ 
tern  of  responses  emerges  for  pay  and  benefits,  and  is 
summarized  in  Figure  5-4.  Here,  a  majority  of  re¬ 
spondents  indicate  that  pay  and  benefits  are  equi¬ 
table,  while  20%  disagree.  There  is  no  upward  trend 
for  the  first  three  administrations.  Again,  it  appears 
that  perceptions  regarding  benefits  reduce  the  num¬ 
ber  of  respondents  perceiving  equity.  Perceptions  of 
benefits  on  the  1986  survey  may  be  related  to  the 
process  of  conversion  from  the  Civil  Service  Retire¬ 
ment  System  (CSRS)  to  the  Federal  Employee  Retire¬ 
ment  System  (FERS).  Figure  5-5  shows  the  response 
rates  for  the  degree  to  which  ATCSs  indicate  work  is 
well  organized.  There  is  an  upward  trend  in  the  agree 
category  from  1984  to  1988,  followed  by  a  drop. 
Finally,  Figure  5-6  also  shows  an  upward  trend  for 
the  respondents  who  report  adequate  management 
direction,  and  a  corresponding  downward  trend  for 
respondents  who  disagree.  Once  again,  in  1990,  the 
percent  of  agree  respondents  drops.  Overall,  there 
appears  to  be  an  improvement  in  perceptions  of 
ATCSs  from  1984  to  1988,  as  reflected  by  increased 
agreement  and  decreased  disagreement  for  most  of 
the  climate  dimensions.  It  is  likely  that  the  1 990  drop 
in  perceptions  for  many  of  the  climate  dimensions  is 
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related  to  the  short-term  budgetary  furloughs  the 
agency  underwent  immediately  prior  to  the  distribu¬ 
tion  of  the  survey. 

Discussion 

This  study  briefly  summarizes  the  perceptions  of 
five  organizational  climate  dimensions  identified  as 
problematic  by  Jones  et  al. ,  and  related  to  the  PATCO 
strike  in  1981,  and  examines  the  FAA’s  interventions 
on  controller  perceptions  over  the  course  of  the 
ATCS  workforce  recovery.  At  a  national  level,  the 
FAA  tried  to  improve  the  selection  and  training  of 
supervisors,  to  enhance  communication  and  under¬ 
standing  through  the  use  of  survey  feedback,  and 
tried  to  create  more  rewarding  jobs  by  altering  pay 
systems.  The  results  suggest  that  these  interventions 
have  had  a  small  positive  impact,  but  have  not  mark¬ 
edly  altered  the  perceptions  of  ATCSs  between  1984 
and  1990.  Relevant  perceptions  did  improve  slightly 
from  1984  to  1988,  then  decreased  slightly  in  1990. 

Of  course,  the  apparent  lack  of  change  must  be 
viewed  with  some  caution  because  of  methodological 
flaws  with  the  present  study.  The  methodological 
problems  include  issues  of  measurement  and  design. 
One  methodological  caveat  is  the  use  of  self-report 
measures.  The  exclusive  reliance  upon  self-report 
measures  can  create  a  susceptibility  to  monomethod 
bias.  Such  a  possibility  is  minimized,  however,  due  to 
the  measures  being  obtained  at  different  times.  A 
second  methodological  problem  is  an  inability  to 
match  respondents  from  one  administration  to  the 
next.  Unmatched  respondents  create  pseudo-inde¬ 
pendent  raters  for  each  administration,  and  thus  it  is 
only  possible  to  examine  agency  trends,  and  not 
change  per  se.  Moreover,  such  analyses  create  more 
conservative  tests,  because  individual  response  biases 
cannot  be  taken  into  account.  In  addition,  the  lack  of 
a  control  group  is  a  design  flaw,  leaving  open  the 
possibility  that  unknown  external  factors  influenced 


employee  perceptions,  either  positively  or  negatively, 
in  addition  to  the  human  resource  initiatives  (e.g., 
changing  retirement  systems  and  furloughs). 

Although  methodological  problems  exist  for  the 
present  study,  there  are  also  several  possible  substan¬ 
tive  explanations  for  why  ATCS  perceptions  have  not 
changed  substantially.  It  is  possible  that  the  interven¬ 
tions  were  not  adequately  developed,  implemented, 
communicated,  or  that  the  interventions  targeted 
inappropriate  organizational  dimensions  or  groups. 
For  example,  the  feedback  from  surveys  was  primarily 
directed  towards  management,  with  management 
then  developing  action  plans  based  on  the  data. 
Employees  generally  received  only  feedback  summa¬ 
ries,  and  the  degree  of  employee  involvement  in 
action  planning  varied  regionally.  Similarly,  the  pay 
demonstration  project  was  not  generalized  to  the 
entire  workforce.  Also,  SIDP  was  eventually  stopped 
because  ratings  of  peers  were  being  conducted  quid 
pro  quo,  making  the  rating  an  ineffective  predictor  of 
supervisory  skills.  Another  possible  explanation  is 
that  the  survey  methodology  used  to  assess  changes  was 
insensitive  to  the  perceptual  changes  that  did  occur. 

Over  the  course  of  rebuilding  the  ATCS  workforce, 
the  influx  of  new  controllers  and  the  human  resources 
initiatives  lead  to  small  improvements  in  the  per¬ 
ceived  organizational  climate.  Small  changes,  as  op¬ 
posed  to  massive  shifts,  and  some  setbacks,  are  to  be 
expected  because  substantially  changing  an  organiza¬ 
tion,  especially  a  large  organization,  is  difficult  and 
time  consuming.  Because  changes  take  place  over 
time  continuous  and  repeated  efforts  are  needed.  Of 
course,  efforts  to  improve  the  FAA  and  the  work 
environment  of  ATCSs  continue  (e.g.,  FAA,  1996; 
General  Accounting  Office,  1996).  Specifically,  re¬ 
cent  changes  in  personnel  and  acquisition  reform  are 
expected  to  aid  in  the  selection  of  new  employees  and 
ease  the  purchase  and  placement  of  new  technologies, 
both  of  which  should  contribute  to  enhanced  climate 
perceptions  of  the  ATCS  workforce. 
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Figure  5-2:  Satisfaction  perceptions  of  air  traffic  control  specialists 


38 


Chapter  5:  Organizational  Climate  Perceptions  of  the  ATCS  Workforce,  1981-7  992 


1984  1986  1988  1990 

Figure  5-4:  Pay  and  benefits  perceptions  of  air  traffic  control  specialists 
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Figure  5-5:  Organization  of  work  perceptions  of  air  traffic  control  specialists 


Figure  5-6:  Management  direction  perceptions  of  air  traffic  control  specialists 
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Table  5-1 :  Sample  Size,  Means  and  Standard  Deviations  for  the  Organization 
Climate  Measures  by  Survey  Administration 


1984 

(N  =12,377) 

1986 

(A/ =14,027) 

1988 

{N  =2,086 ) 

1990 

{N  =2,456 ) 

Organization  Climate  Measures 

M 

SD 

M 

SD 

M 

SD 

M 

SD 

Job  Satisfaction^ 

3.32 

1.25 

3.37 

1.23 

3.64 

1.15 

3.49 

1.18 

Satisfaction 

3.32 

1.25 

3.26 

0.80 

3.35 

0.78 

3.22 

0.78 

Management  Concern  for 
Employees 

2.31 

0.99 

2.49 

1.04 

2.57 

0.98 

2.42 

0.96 

Pay  and  Benefits 

3.49 

0.89 

3.12 

0.76 

3.32 

0.71 

3.20 

0.75 

Organization  of  Work 

2.53 

0.85 

2.69 

0.91 

2.75 

0.85 

2.65 

0.85 

Management  Direction 

2.92 

0.83 

2.98 

0.87 

3.04 

0.84 

2.92 

0.85 

Notes:  ^Based  on  a  single  item 
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CURRENT  FAA  CONTROLLER  WORKFORCE  DEMOGRAPHICS,  FUTURE 
REQUIREMENTS,  AND  RESEARCH  QUESTIONS 

David  J,  Schroeder,  Dana  Broach,  and  William  L.  Farmer 

FAA  Civil  Aeromedical  Institute,  Oklahoma  City,  Oklahoma 

Between  1981  and  1992^  the  Federal  Aviation  Administration  replaced  approximately  11,500  air  traffic 
controllers  in  the  wake  of  the  1981  Professional  Air  Traffic  Controller  Organization  (PATCO)  strike.  Three  issues 
are  considered  in  this  paper:  the  aging  controller  workforce  after  the  year 2000,  trainingfor  advanced  air  traffic  control 
technologies  with  an  aging  workforce,  and  recruitment  and  selection  criteria  for  future  controllers.  Ongoing  research 
to  address  these  issues  are  described. 


Following  the  Professional  Air  Traffic  Controller 
Organization  (PATCO)  strike  of  1981,  the  Federal 
Aviation  Administration  (FAA)  was  tasked  with  re¬ 
placing  approximately  10,400  air  traffic  controllers. 
As  the  point  of  retirement  eligibility  for  the  first  of 
these  replacement  controllers  approaches,  a  consider¬ 
able  amount  of  resources  will  need  to  be  invested  in 
planning  for  and  initiating  recruitment,  selection, 
and  training  programs  for  the  inevitable  influx  of  new 
controllers.  As  a  result,  the  FAA  faces  a  number  of 
personnel-related  research  and  operational  questions. 
This  paper  focuses  on  three  issues: 

•  Aging  of  the  Air  T  raffic  Control  Specialist  (AT  CS) 
workforce; 

•Employee  training  for  the  advanced  National 
Airspace  System  (NAS)  air  traffic  control  (ATC) 
system  technologies;  and 

•The  recruitment  and  selection  of  personnel  to 
operate  the  more  automated  ATC  system  of  the 
future. 

Earlier  papers  in  this  volume  focused  on  the  pro¬ 
cesses  and  tools  used  to  rebuild  the  controller 
workforce  after  the  1981  strike.  But  as  the  present 
workforce  ages,  and  inexorably  moves  toward  retire¬ 
ment  eligibility,  the  issue  of  future  selection  to  re¬ 
plenish  the  ATC  system  becomes  critical.  Combined 
with  this  are  the  soon  to  be  changing  demands  im¬ 
posed  by  the  evolving  NAS  (FAA,  1996).  A  large 
amount  of  research  has  gone  into  defining  the  knowl¬ 
edge,  skills,  and  abilities  of  the  ideal  air  traffic 


controller.  As  system  dynamics  and  technologies 
change,  many  of  those  essentials  that  define  perfor¬ 
mance  in  today's  system  may  no  longer  be  as  impor¬ 
tant  in  tomorrow's  job.  With  this  in  mind,  current 
efforts  are  focused  on  defining  the  new  system  and 
the  skill  requirements  of  the  future  controller 
workforce. 

Aging  of  the  ATCS  Workforce 

The  vast  majority  (87.9%)  of  the  current  ATCS 
workforce  entered  the  FAA  following  the  PATCO 
strike  of  1981.  To  better  understand  the  characteris¬ 
tics  of  the  ATC  population,  we  extracted  information 
regarding  its  age  and  experience  distributions  from 
the  Consolidated  Personnel  Management  Informa¬ 
tion  System  (CPMIS)  maintained  by  the  FAA  (Fig¬ 
ure  6-1).  The  sample  of  14,569  ATCSs  (as  of  April 
1996)  does  not  include  personnel  who  were  listed  in 
ATCS  office,  staff,  or  supervisory/management  posi¬ 
tions.  Since  a  significant  portion  of  the  workforce 
was  hired  in  the  decade  following  the  ATC  strike  of 
1981,  we  found  the  expected  bulge  of  controllers 
(62.  7%)  in  the  30-39  age  range.  The  average  age  of 
the  current  workforce  is  37,  with  a  standard  deviation 
of  6.38  years.  Of  this  sample  of  14,569  active  con¬ 
trollers,  1 1 .3%  of  the  controllers  were  40  years  of  age 
and  older,  and  they  had  at  least  1 5  years’  experience. 
The  percentage  of  the  post-strike  group  that  will  fall 
in  that  same  category  in  2001  increases  to  40%.  By 
2011,  nearly  the  entire  population  of  the  current 
workforce  will  either  be  in  that  group,  or  will  have 
already  retired. 
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Projections  of  the  staffing  requirements  and  asso¬ 
ciated  air  traffic  levels  are  presented  in  Figure  6-2.  A 
gradual  increase  in  ATCS  staffing  over  the  next  two 
decades  is  designed,  in  part,  to  keep  pace  with  the 
expected  increase  in  aviation  activity  (Air  Traffic 
Services,  1996).  To  determine  the  effects  of  the 
continued  aging  of  the  current  group  of  controllers 
on  the  need  to  hire  and  train  new  personnel,  we 
developed  a  research  model  that  allows  us  to  forecast 
the  number  of  ATCSs  eligible  for  retirement  over  the 
next  decade.  Recognizing  that  there  are  many  factors 
that  can  be  incorporated  into  a  model  for  projecting 
the  hiring  and  training  flow  of  the  future,  ours  is 
sensitive  both  to  attrition  associated  with  deaths  due 
to  accidents  and  other  health  conditions  (this  is  not 
age  corrected  so  it  represents  a  conservative  estimate 
of  losses  attributed  to  these  events)  and  the  availabil¬ 
ity  of  retirement  eligibility  at  age  56.  While  different 
models  may  produce  slightly  different  predictions, 
the  general  shape  of  the  distributions  remain  rela¬ 
tively  the  same.  Using  this  research  model,  we  pro¬ 
jected  the  number  of  retirement-eligible  controllers 
from  2001  through  2020,  as  illustrated  in  Figure  6- 
3.  The  greatest  potential  loss  due  to  retirements  of 
ATCSs  will  occur  around  2015  to  2017.  By  2016, 
over  50%  oftoday's  AT C  workforce  will  have  reached 
the  age  at  which  they  are  eligible  for  retirement. 

The  General  Accounting  Office  (GAO)  was  criti¬ 
cal  in  a  recent  report  (GAO,  1997)  of  the  FAA's 
retirement  projections  that  were  based  on  the  as¬ 
sumption  that,  in  any  year,  around  20%  of  those 
eligible  would  retire.  The  GAO  developed  projec¬ 
tions  that  were  based  on  recent  data  that  indicated 
that  the  average  controller  retires  at  56  years  of  age 
with  31  years  of  service.  When  compared  with  the 
FAA  projections,  GAO  predicted  smaller  numbers  of 
retirees  in  1997  through  2001.  GAO  based  their 
calculations  on  the  probabilities  of  retirement  of 
controllers  in  various  age  cohorts  during  fiscal  years 
1992-1996.  The  average  yearly  retirements  would 
increase  from  198  in  1997  to  299  in  2003.  In  2009, 
more  than  600  retirements  would  be  expected.  That 
average  number  of  yearly  retirements  is  likely  to 
remain  at  that  level  or  higher  for  several  years,  given 
the  large  numbers  of  ATCSs  who  will  reach  age  56 
from  year  2009  through  2017. 


Controller  Aging  and  Performance 

Another  consideration  associated  with  the  overall 
aging  of  the  workforce  during  the  first  two  decades  of 
the  21st  century  concerns  the  potential  effects  of 
aging  on  cognitive  functioning  and  the  consequences 
of  those  changes  for  on-the-job  performance.  There 
are  indications  in  the  scientific  literature  that  during 
the  middle  decades  of  life  most  individuals  experi¬ 
ence  some  decline  in  the  speed  of  their  cognitive 
functioning.  In  fact,  Fozard  and  Nutall  (1971),  in 
their  study  of  age-related  changes  in  scores  on  the 
General  Aptitude  Test  Battery  (GATB),  indicated 
that  the  average  60  year-old  individual  would  be  able 
to  meet  the  occupational  aptitude  patterns  for  only 
17  of  36  occupations.  Changes  are  more  frequently 
observed  in  the  fluid  or  process  aspects  of  cognition 
(speed  of  processing  information,  memory,  and  mak¬ 
ing  decisions).  Some  scientists  feel  that  overall  per¬ 
formance  on  those  tasks  may  start  to  decline  at  ages 
as  early  as  in  the  20s  (Salthouse,  1990).  However, 
what  is  often  referred  to  as  the  crystallized  measures 
of  intelligence  (vocabulary  and  general  information) 
may  improve  with  increasing  age  until  the  late  60s. 
When  viewed  in  the  context  of  most  jobs,  Schaie 
(1988)  has  argued  that  the  reported  age  differences 
are  generally  too  small  to  have  meaningful  conse¬ 
quences.  However,  little  has  been  said  about  the 
potential  implications  of  these  changes  for  personnel 
in  safety- related  positions  where  speed  of  informa¬ 
tion  processing,  memory,  and  decision-making  are  of 
critical  iniportance. 

Nearly  all  of  the  scientific  information  available 
refers  to  the  effects  of  age-related  declines  in  perfor¬ 
mance  on  somewhat  novel  laboratory  tasks,  com¬ 
pared  with  more  familiar  tasks  or  job  performance.  A 
series  of  studies  by  Trites,  Cobb,  and  their  associates 
(e.g.,  Cobb,  Lay,  &  Bourdet,  1971;  Trites,  1963; 
Trites  &  Cobb,  1964a;  1964b)  consistently  demon¬ 
strated  an  inverse  relationship  between  age  of  entry 
and  performance  of  ATCS  trainees  at  the  FAA  Acad¬ 
emy  and  in  field  training.  Those  studies  led  to  Con¬ 
gressional  action  to  restrict  the  age  of  entry  into  the 
FAA  Academy  training  to  age  30. 

A  subsequent  Civil  Aeromedical  Institute  (CAMI) 
study  (VanDeventer  &  Baxter,  1984)  of  8,573  FAA 
Academy  trainees  where  the  age  of  entry  was  more 
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restrictive  (18-30)  than  that  studied  by  Cobb  and 
associates,  demonstrated  the  typical  inverse  relation¬ 
ship  between  age  and  pass  rate.  Figure  6-4  presents 
the  outcomes  in  the  FAA  Academy  screening  pro¬ 
grams  for  27,925  entrants  from  1981  through  1992. 
Overall  success  in  the  FAA  Academy  declined  from 
66.6%  of  those  age  22  and  younger  to  53.7%  of  those 
aged  27-28,  and  48.6%  of  those  in  the  31  or  older  age 
category.  Age-related  changes  were  evident  in  both 
the  failure  rates  as  well  as  withdrawals.  Lowered 
success  rates  were  also  evident  in  field  training.  Out¬ 
comes  for  the  FAA  Academy  graduates  initially  as¬ 
signed  to  an  en  route  ATC  facility  for  training  are 
presented  in  Figure  6-5.  To  be  consistent  with 
Manning’s  (this  volume)  analysis  of  OJT  outcomes, 
controllers  still  in  training,  or  who  left  training  for 
reasons  other  than  performance,  were  excluded  from 
this  analysis.  While  80.7%  of  those  age  22  or  younger 
successfully  reached  the  full  performance  level  at 
their  first  en  route  facility,  only  52.8%  of  those  in  the 
oldest  age  group  (31+)  did.  From  the  younger  through 
older  age  groups,  there  was  a  systematic  increase  in 
the  percentages  of  trainees  who  switched  to  another 
option  (Terminal  or  Flight  Service  Station)  or  were 
terminated  from  training. 

The  overall  success  rates  of  trainees  entering  the 
terminal  option  were  considerably  higher,  ranging 
from  88%  for  those  aged  22  or  younger  to  78.6%  of 
those  aged  3 1  and  older.  One  indication  of  the  lower 
overall  difficulty  of  the  terminal  option  on-the-job 
training  program  is  the  time  required  to  reach  full 
performance  level  (FPL)  status.  It  ranges  from  slightly 
over  one  year  at  the  level  1-2  terminals  to  approxi¬ 
mately  2  1/2  years  at  a  level  4  or  5  facility.  The  latter 
is  still  below  the  average  of  slightly  more  than  three 
years  required  at  the  en  route  centers.  Much  of  the 
observed  difference  in  success  rates  across  ages  can  be 
attributed  to  outcomes  from  the  larger  terminal  fa¬ 
cilities  (Level  4  and  5).  Percentages  of  post-strike 
trainees  who  completed,  switched  options,  or  washed 
out  of  field  training  at  the  level  4  and  5  terminal 
facilities  are  presented  in  Figure  6-6,  again  excluding 
those  still  in  training  or  who  left  for  reasons  unrelated 
to  job  performance.  The  percentage  who  reached  FPL 
status  ranged  from  76.9%  of  controllers  in  the  22 
years  of  age  or  younger  group  to  46.3%  of  those  in  the 
31  and  older  age  group.  Thus,  in  the  more  demand¬ 
ing  terminal  environments,  there  is  evidence  of  the 
expected  age-related  decline  in  on-the-job  training 


success.  Most  of  the  trainees  at  level  5  terminal 
facilities  are  selected  from  the  ranks  of  FPL  control¬ 
lers  at  other  facilities.  These  outcomes  are  again 
consistent  with  historical  data  gathered  from  respec¬ 
tive  field  facilities  (Trites,  1963;  Trites  &  Cobb, 
1964a,  1964b).  Thus,  older  adults  experience  greater 
difficulty  completing  the  FAA  Academy  screening 
and  on-the-job  training  programs  for  Air  Traffic 
Control  Specialists. 

Some  scientists  have  argued  that  we  are  not  likely 
to  observe  age-related  differences  at  work  because 
many  of  the  tasks  are  in  continuous  use  and  are  thus 
highly  over-learned.  It  is  difficult  to  assess  the  extent 
to  which  older  employees  may  be  less  proficient  at 
work  due  to  the  multitude  of  factors  that  influence 
attrition  as  employees  age  (poorer  workers  often  drop 
out  or  move  to  other  jobs,  and  the  best  are  often 
selected  for  promotions  or  are  given  opportunities  to 
switch  to  more  challenging  jobs).  The  effects  of 
seniority  on  job  assignments  in  many  occupational 
settings  are  also  often  poorly  understood.  Salthouse 
(1990)  describes  several  mechanisms  that  may  serve 
to  preserve  overall  competency  and  job  performance 
as  adults  experience  age-related  declines  in  the  effi¬ 
ciency  of  basic  processes:  compensation,  accommo¬ 
dation,  elimination,  and  compilation.  Rhodes  (1983), 
in  a  review  of  age-related  differences  in  work  atti¬ 
tudes  and  behavior,  concluded  that  the  outcomes 
from  studies  of  the  relationship  between  age  and  job 
performance  are  mixed.  Older  employees  did  express 
higher  levels  of  job  satisfaction,  satisfaction  with  the 
work  itself,  and  job  involvement. 

Evidence  for  a  possible  moderating  effect  of  expe¬ 
rience  on  the  typical  effects  of  age  on  job-related  tasks 
is  mixed.  General  support  was  found  by  Salthouse  for 
typing  tasks  (1990)  and  by  Morrow,  Leirer,  Altieri, 
and  Fitzsimmons  (1994)  on  readback  of  visual  or 
vocally-presented  ATC  communications.  Evidence 
was  more  limited  for  a  set  of  time-sharing  tasks 
involving  groups  of  pilots  and  non-pilots  (Tsang  & 
Voss,  1996).  Salthouse,  Babcock,  Skovronek, 
Mitchell,  &  Palmon  (1990)  found  little  support 
among  groups  of  practicing  architects  on  spatial 
visualization  tasks.  Thus,  while  there  is  some  labora¬ 
tory-based  evidence  to  support  the  critical  role  of 
expertise  or  experience  in  the  performance  of  older 
workers  on  job-related  tasks,  there  is  insufficient  job- 
related  performance  research  available  to  clearly  docu¬ 
ment  the  importance  of  various  factors  in  determining 
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the  job  performance  of  older  workers,  particularly  in 
safety  critical  occupations.  However,  Cobb  and  asso¬ 
ciates  (Cobb,  1967;  Cobb,  Nelson,  &  Matthews, 
1973)  have  demonstrated  that  combined  ratings  (su¬ 
pervisors,  crew  chiefs,  and  coworkers)  of  controller 
performance  are  lower  for  older  than  for  younger 
controllers.  This  is  consistent  with  data  gathered 
earlier  at  the  en  route  centers.  There  are  several 
factors  (motivation,  differences  in  aptitudes  and  abili¬ 
ties,  and  rating  bias)  that  may  have  contributed  to 
these  differences,  other  than  physiological  aging. 
While  we  do  not  understand  the  full  implications  of 
the  age-related  changes  on  the  performance  of  air 
traffic  controllers,  careful  attention  should  be  paid  to 
what  appears  to  be  age-related  performance  changes 
as  the  overall  age  of  the  controller  workforce  increases 
during  the  next  15  years. 

Training  Implications 

Based  on  the  projected  increase  in  retirement  eli¬ 
gible  personnel  and  the  lead-time  of  around  two  to 
three  years  for  individuals  to  complete  training,  the 
FAA  is  in  the  process  of  developing  plans  to  hire 
controllers  over  the  next  two  decades.  Facilities  will 
thus  have  expanding  training  requirements  over  the 
next  20+  years  to  equip  new  controllers  to  handle 
traffic  in  the  future,  as  older  controllers  retire.  How¬ 
ever,  this  is  not  the  only  training  burden  faced  by 
facility  managers  and  the  Air  Traffic  Service.  Within 
the  next  20  years,  enhanced  hardware/ software  ATC 
work  stations,  decision  aids,  and  new  policies  and 
procedures  will  be  introduced  into  the  NAS  to  im¬ 
prove  capacity  and  to  increase  safety  and  efficiency. 
By  the  turn  of  the  century  we  will  observe  the  effects 
of  the  introduction  of  data  link,  the  ARTCC  Display 
System  Replacement  (DSR),  Standard  Tower  Auto¬ 
mation  Replacement  System  (STARS),  Global  Navi¬ 
gation  Satellite  System  (GNSS)  technologies,  and 
airborne  warning  technologies.  During  the  first  de¬ 
cade  of  the  21st  century,  new  decision  support  sys¬ 
tems  will  be  introduced.  Emphasis  on  integrating 
“free  flight”  concepts  into  the  NAS  may  result  in  a 
number  of  significant  changes  in  how  air  traffic  is 
handled.  Under  some  proposals  there  may  be  a  shift 
in  responsibility  for  separation  of  aircraft  between 
flight  crew  members  and  controllers,  as  a  part  of  “free 
flight.”  No  matter  which  scenario  is  finally  adopted, 
the  introduction  of  these  new  systems  and  procedures 
into  the  NAS  over  the  next  two  decades  will  generate 
continuing  requirements  to  train  ATCSs  to  operate 


these  new  systems.  Since  these  changes  will  largely 
coincide  with  the  influx  of  new  controllers,  it  will 
present  a  significant  logistical  effort  with  respect  to 
retraining  the  existing  cadre  of  full  performance  level 
controllers,  while  training  new  controllers  at  the 
same  time.  Concerns  about  the  availability  of  suffi¬ 
cient  staff  within  the  facility  to  maintain  normal 
operations  and  still  provide  on-the-job  training  are 
likely  to  be  of  special  concern  at  facilities  that  have 
been  identified  as  “hard-to-staff,”  where  staffing  lev¬ 
els  have  remained  below  projected  levels.  Thus,  there 
will  be  a  general  need  for  facility-specific  plans  to 
address  the  training  requirements  associated  with  the 
hiring  of  new  controllers  to  replace  retirees  and  the 
retraining  of  the  existing  workforce  to  function  on 
the  more  automated  systems  of  the  future. 

While  the  cognitive  changes  in  performance  capa¬ 
bilities  may  not  pose  a  significant  problem  for  the  safe 
and  expeditious  flow  of  traffic  under  typical  condi¬ 
tions,  it  may  pose  additional  burdens  as  older  ATCSs 
are  required  to  transition  to  new  technologies  and 
new  procedures.  Evidence  suggests  that  older  em¬ 
ployees  experience  greater  difficulty  acquiring  new 
skills  than  do  younger  employees  (Charness  & 
Bosman,  1990).  In  the  use  of  computer  technology, 
there  is  research  that  suggests  that  older  adults  re¬ 
quire  longer  training  time  and,  in  general,  do  not 
obtain  the  same  level  of  skill  as  younger  individuals 
(Czaja,  1996;  Shark  &  Czaja,  1994).  The  difficulty 
of  this  transition  is,  of  course,  dependent  on  the 
extent  to  which  the  job  changes  and  the  nature  of 
those  changes.  At  this  point,  we  do  not  know  enough 
about  the  proposed  future  systems  to  determine  the 
extent  to  which  age-related  factors  should  be  consid¬ 
ered  in  the  development  of  the  new  procedures  and 
training  programs.  Human  factors  assessments  will 
be  needed  to  determine  the  appropriateness  of  training 
approaches  for  transitioning  to  the  new  technologies. 

Selecting  Controllers  for  Future  Systems 

Presentations  made  earlier  in  this  session  have 
illustrated  efforts  associated  with  the  recruitment, 
selection,  and  training  of  new  ATCSs.  Research  is 
currently  ongoing  with  respect  to  developing  and 
validating  a  3'^  generation  computer-administered 
test  battery  to  select  controllers  for  the  near  term 
(1998  -  2005).  But  what  are  the  cognitive  and  per¬ 
sonal  characteristics  that  may  be  needed  by  control¬ 
lers  operating  future  systems?  The  new  technologies 
will  involve  greater  automation  and  the  availability 
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of  increased  decision  aiding.  The  extent  to  which  the 
new  system  and  procedures  impact  on  the  nature  of 
the  job  is  highly  dependent  on  the  level  of  automa¬ 
tion  selected  (Wickens,  Mavor,  &  McGee,  1997).  In 
his  recent  book  on  automation  in  aviation,  Billings 
(1997)  provides  four  scenarios  regarding  the  nature 
of  the  future  air  traffic  control  system:  management 
by  delegation;  management  by  consent;  management 
by  exception;  and  free  flight.  There  are  marked  differ¬ 
ences  in  the  roles  and  responsibilities  of  ATCSs 
under  these  scenarios.  Both  the  Automated  En  Route 
air  traffic  control  (AERA)  and  free  flight  concepts 
appear  to  raise  a  number  of  human  factors  concerns 
related  to  controller-machine  interactions. 

CAMI  scientists  have  embarked  on  a  program  of 
research  designed  to  provide  an  answer  to  those 
questions.  The  initial  study  was  designed  to  assess 
controller  perceptions  of  AERA  and  determine  how 
perceptions  of  task  activities  under  AERA  would 
impact  the  requisite  knowledge,  skills,  and  abilities 
of  ATCSs.  Findings  by  Manning  &  Broach  (1993) 
indicate  that  many  of  the  same  skills  and  abilities 
might  be  required.  Since  we  are  no  longer  moving 
down  an  identical  technological  pathway,  the  ques¬ 
tion  remains  regarding  the  demands  and  job  tasks 
associated  with  the  future  ATC  systems.  The  infra¬ 
structure  technologies  that  are  being  readied  for 
introduction  into  the  NAS  during  the  remainder  of 
this  decade  are  not  likely  to  dramatically  alter  the  job 
duties  and  functions  of  ATCSs.  However,  the  imple¬ 
mentation  of  the  initial  and  upgraded  versions  of 
decision-support  systems,  and  the  revised  AERA  sys¬ 
tem  are  likely  to  substantially  modify  the  nature  of 
the  job.  Given  the  time  required  to  recruit,  select,  and 
train  personnel,  a  new  advanced  selection  system 
needs  to  be  developed  coincident  with  the  develop¬ 
ment  of  the  proposed  future  systems. 

Traditional  approaches  to  job  analysis  have  fo¬ 
cused  on  jobs  that  incumbents  have  already  been 
performing.  In  planning  for  the  future,  an  approach 
is  needed  that  will  allow  us  to  identify  the  knowledge, 
skills  and  abilities  required  as  the  systems  are  under 
development.  One  strategy  is  to  utilize  the  “strategic 
job  analysis”  approach  presented  by  Schneider  and 
Konz  (1989).  Efforts  are  underway  at  CAMI  to 
develop  the  methodology  required  to  conduct  a  stra¬ 


tegic  job  analysis  for  both  air  traffic  control  and 
airway  facilities  positions.  Once  developed,  the  ap¬ 
proach  will  be  used  to  assess  the  changes  in  knowl¬ 
edge,  skills,  and  abilities  as  new  systems  are  developed 
for  the  NAS.  With  the  anticipated  increased  empha¬ 
sis  on  coordination  and  shared  decision-making, 
there  may  be  additional  requirements  to  identify 
personality  dimensions  that  play  a  critical  role  in 
teamwork.  Of  course  if  the  level  of  automation  se¬ 
lected  is  one  where  the  human  operator  is  primarily 
a  system  monitor  and  will  only  take  over  in  the  advent 
of  an  automation-related  failure,  this  presents  a  much 
different  view  of  the  nature  of  the  job  and  the  neces¬ 
sary  knowledge,  skills  and  abilities.  Outcomes  from 
the  future  job  analysis  assessments  will  be  compared 
with  the  recent  baseline  job  analysis  data  (Nickels, 
Bobko,  Blair,  Sands,  &  Tartak,  1995).  The  identifi¬ 
cation  of  the  requisite  knowledge,  skills,  and  abilities 
that  are  sufficiently  different  from  those  required 
with  current  systems  will  be  used  to  modify  existing 
selection  and  training  programs.  Ideally,  this  will 
allow  the  FAA  to  develop  an  integrated  recruitment, 
selection,  training,  and  job  performance  data  collec¬ 
tion  and  analysis  system.  That  system  should  be  in 
place  in  time  to  select  the  personnel  who  will  be 
operating  those  future  systems.  An  outline  for  re¬ 
search  to  support  selection  of  personnel  for  the  future 
NAS  architecture  is  described  in  Broach  (1997). 

Summary 

While  Billings  (1997)  and  Wickens,  Mavor,  and 
McGee  (1997)  have  reviewed  and  identified  a  num¬ 
ber  of  human  factors  that  need  to  be  addressed  during 
the  transition  to  the  more  automated  ATC  system  of 
the  future,  in  this  paper  we  have  sought  to  identify 
personnel-related  issues  and  concerns.  The  projected 
future  increase  in  air  traffic,  aging  of  the  ATC 
workforce,  and  increased  requirements  to  train  both 
new  personnel  and  members  of  the  existing  workforce 
to  operate  the  future  more  automated  ATC  system, 
will  need  to  be  addressed.  With  the  advent  of  the  new 
technologies,  there  is  also  a  clear  need  to  identify  the 
knowledge,  skills  and  abilities  required  to  perform 
the  duties  of  the  future  controller  and  to  develop 
instruments  to  support  the  recruitment  and  selection 
of  individuals  who  can  best  operate  those  systems. 
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Figure  6-2:  Projected  controller  workforce  (CWF)  vs.  aviation  activity  (1982-2002). 
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Figure  6-4:  FAA  Academy  outcomes  by  age  group  for  all  entrants,  1 981-1 992 


Figure  6-5:  OJT  outcomes  at  first  facility  by  age  category  for  FAA  Academy  graduates 
initially  assigned  to  en  route  centers  (N  =  8,748) 
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Figure  6-6:  OJT  outcomes  at  first  facility  by  age  category  for  FAA  Academy  graduates  initially  assigned 

to  Level  4  and  5  terminals  (A= 1,050) 
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